INFO: Downloading File to /root/PM-DARTS2/...

Succeed: Total num: 37, size: 170,637,559. OK num: 37(download 37 objects).

average speed 226610000(byte/s)

0.756404(s) elapsed
INFO: Downloading succeed.
Network is under initialization...
Network successfully initialized.
WARN: ./requirements.txt not found, skip installing requirements.
Training with a single process on 1 GPUs.
Data processing configuration for current model + dataset:
	input_size: (3, 32, 32)
	interpolation: bilinear
	mean: (0.49139968, 0.48215827, 0.44653124)
	std: (0.24703233, 0.24348505, 0.26158768)
	crop_pct: 1.0
	crop_mode: center

-------------------------------
Learnable parameters
Student: 0.67M
Extra: 0.00M
-------------------------------
Scheduled epochs: 50
p_max: 0.25
search_space = s2
Using downloaded and verified file: /mnt/PM-DARTS2/data/cifar-10-python.tar.gz
Extracting /mnt/PM-DARTS2/data/cifar-10-python.tar.gz to /mnt/PM-DARTS2/data
Train: 0 [   0/390]  Loss: 2.392 (2.39)  Acc@1: 12.5000 (12.5000)  Acc@5: 42.1875 (42.1875)LR: 2.500e-02
Train: 0 [  50/390]  Loss: 2.127 (2.04)  Acc@1: 26.5625 (24.8162)  Acc@5: 81.2500 (77.0833)LR: 2.500e-02
Train: 0 [ 100/390]  Loss: 1.929 (1.96)  Acc@1: 29.6875 (26.7481)  Acc@5: 76.5625 (80.4301)LR: 2.500e-02
Train: 0 [ 150/390]  Loss: 1.505 (1.89)  Acc@1: 43.7500 (29.4495)  Acc@5: 96.8750 (82.5642)LR: 2.500e-02
Train: 0 [ 200/390]  Loss: 1.650 (1.83)  Acc@1: 45.3125 (31.2578)  Acc@5: 89.0625 (84.2351)LR: 2.500e-02
Train: 0 [ 250/390]  Loss: 1.596 (1.79)  Acc@1: 34.3750 (33.3167)  Acc@5: 93.7500 (85.5640)LR: 2.500e-02
Train: 0 [ 300/390]  Loss: 1.322 (1.74)  Acc@1: 56.2500 (34.9927)  Acc@5: 95.3125 (86.5500)LR: 2.500e-02
Train: 0 [ 350/390]  Loss: 1.641 (1.71)  Acc@1: 40.6250 (36.5296)  Acc@5: 89.0625 (87.0905)LR: 2.500e-02
Train: 0 [ 390/390]  Loss: 1.577 (1.68)  Acc@1: 40.0000 (37.4920)  Acc@5: 97.5000 (87.6440)LR: 2.500e-02
train_acc 37.492000
Valid: 0 [   0/390]  Loss: 1.645 (1.65)  Acc@1: 35.9375 (35.9375)  Acc@5: 87.5000 (87.5000)
Valid: 0 [  50/390]  Loss: 1.325 (1.44)  Acc@1: 54.6875 (46.0784)  Acc@5: 96.8750 (92.3713)
Valid: 0 [ 100/390]  Loss: 1.313 (1.41)  Acc@1: 48.4375 (47.2772)  Acc@5: 95.3125 (92.7135)
Valid: 0 [ 150/390]  Loss: 1.368 (1.43)  Acc@1: 59.3750 (47.1337)  Acc@5: 92.1875 (92.4979)
Valid: 0 [ 200/390]  Loss: 1.443 (1.43)  Acc@1: 42.1875 (47.3336)  Acc@5: 92.1875 (92.4440)
Valid: 0 [ 250/390]  Loss: 1.591 (1.43)  Acc@1: 45.3125 (47.3108)  Acc@5: 89.0625 (92.5921)
Valid: 0 [ 300/390]  Loss: 1.336 (1.43)  Acc@1: 46.8750 (47.3630)  Acc@5: 98.4375 (92.6287)
Valid: 0 [ 350/390]  Loss: 1.324 (1.42)  Acc@1: 50.0000 (47.3558)  Acc@5: 93.7500 (92.6238)
Valid: 0 [ 390/390]  Loss: 1.386 (1.42)  Acc@1: 45.0000 (47.3920)  Acc@5: 92.5000 (92.6720)
valid_acc 47.392000
epoch = 0   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4915, 0.5085],
        [0.4928, 0.5072],
        [0.4875, 0.5125],
        [0.4856, 0.5144],
        [0.4877, 0.5123],
        [0.4895, 0.5105],
        [0.4845, 0.5155],
        [0.4849, 0.5151],
        [0.4848, 0.5152],
        [0.4916, 0.5084],
        [0.4865, 0.5135],
        [0.4867, 0.5133],
        [0.4865, 0.5135],
        [0.4818, 0.5182]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4984, 0.5016],
        [0.4958, 0.5042],
        [0.4978, 0.5022],
        [0.4940, 0.5060],
        [0.4967, 0.5033],
        [0.4948, 0.5052],
        [0.4975, 0.5025],
        [0.4940, 0.5060],
        [0.4924, 0.5076],
        [0.4950, 0.5050],
        [0.4983, 0.5017],
        [0.4899, 0.5101],
        [0.4922, 0.5078],
        [0.4904, 0.5096]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 1 [   0/390]  Loss: 1.413 (1.41)  Acc@1: 40.6250 (40.6250)  Acc@5: 93.7500 (93.7500)LR: 2.498e-02
Train: 1 [  50/390]  Loss: 1.270 (1.36)  Acc@1: 51.5625 (50.1532)  Acc@5: 92.1875 (94.2708)LR: 2.498e-02
Train: 1 [ 100/390]  Loss: 1.370 (1.34)  Acc@1: 54.6875 (50.9746)  Acc@5: 92.1875 (94.0285)LR: 2.498e-02
Train: 1 [ 150/390]  Loss: 1.241 (1.32)  Acc@1: 53.1250 (51.6142)  Acc@5: 98.4375 (94.2260)LR: 2.498e-02
Train: 1 [ 200/390]  Loss: 1.144 (1.31)  Acc@1: 62.5000 (52.0522)  Acc@5: 95.3125 (94.3175)LR: 2.498e-02
Train: 1 [ 250/390]  Loss: 1.454 (1.29)  Acc@1: 45.3125 (53.1997)  Acc@5: 87.5000 (94.5530)LR: 2.498e-02
Train: 1 [ 300/390]  Loss: 1.033 (1.27)  Acc@1: 59.3750 (54.0334)  Acc@5: 96.8750 (94.6532)LR: 2.498e-02
Train: 1 [ 350/390]  Loss: 1.143 (1.25)  Acc@1: 62.5000 (54.6519)  Acc@5: 96.8750 (94.7427)LR: 2.498e-02
Train: 1 [ 390/390]  Loss: 0.9140 (1.24)  Acc@1: 70.0000 (55.0520)  Acc@5: 97.5000 (94.8440)LR: 2.498e-02
train_acc 55.052000
Valid: 1 [   0/390]  Loss: 1.158 (1.16)  Acc@1: 59.3750 (59.3750)  Acc@5: 96.8750 (96.8750)
Valid: 1 [  50/390]  Loss: 0.9762 (1.12)  Acc@1: 62.5000 (57.9044)  Acc@5: 98.4375 (96.2316)
Valid: 1 [ 100/390]  Loss: 0.9893 (1.14)  Acc@1: 67.1875 (58.1683)  Acc@5: 95.3125 (95.7921)
Valid: 1 [ 150/390]  Loss: 1.166 (1.15)  Acc@1: 57.8125 (57.9884)  Acc@5: 100.0000 (95.7678)
Valid: 1 [ 200/390]  Loss: 1.158 (1.15)  Acc@1: 59.3750 (57.8825)  Acc@5: 95.3125 (95.6701)
Valid: 1 [ 250/390]  Loss: 1.206 (1.16)  Acc@1: 56.2500 (57.7627)  Acc@5: 98.4375 (95.7794)
Valid: 1 [ 300/390]  Loss: 1.124 (1.15)  Acc@1: 57.8125 (57.8177)  Acc@5: 95.3125 (95.7797)
Valid: 1 [ 350/390]  Loss: 1.014 (1.15)  Acc@1: 65.6250 (58.0084)  Acc@5: 100.0000 (95.7309)
Valid: 1 [ 390/390]  Loss: 1.150 (1.15)  Acc@1: 62.5000 (58.0600)  Acc@5: 92.5000 (95.7240)
valid_acc 58.060000
epoch = 1   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4902, 0.5098],
        [0.4865, 0.5135],
        [0.4834, 0.5166],
        [0.4821, 0.5179],
        [0.4776, 0.5224],
        [0.4870, 0.5130],
        [0.4805, 0.5195],
        [0.4776, 0.5224],
        [0.4761, 0.5239],
        [0.4863, 0.5137],
        [0.4838, 0.5162],
        [0.4750, 0.5250],
        [0.4734, 0.5266],
        [0.4699, 0.5301]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4997, 0.5003],
        [0.4957, 0.5043],
        [0.4956, 0.5044],
        [0.4861, 0.5139],
        [0.4928, 0.5072],
        [0.4868, 0.5132],
        [0.4936, 0.5064],
        [0.4885, 0.5115],
        [0.4885, 0.5115],
        [0.4921, 0.5079],
        [0.4953, 0.5047],
        [0.4867, 0.5133],
        [0.4891, 0.5109],
        [0.4817, 0.5183]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 2 [   0/390]  Loss: 1.098 (1.10)  Acc@1: 53.1250 (53.1250)  Acc@5: 96.8750 (96.8750)LR: 2.491e-02
Train: 2 [  50/390]  Loss: 0.9564 (1.08)  Acc@1: 67.1875 (60.8762)  Acc@5: 93.7500 (96.2623)LR: 2.491e-02
Train: 2 [ 100/390]  Loss: 0.9562 (1.07)  Acc@1: 62.5000 (61.3861)  Acc@5: 100.0000 (96.2407)LR: 2.491e-02
Train: 2 [ 150/390]  Loss: 0.9210 (1.07)  Acc@1: 60.9375 (61.5273)  Acc@5: 100.0000 (96.2438)LR: 2.491e-02
Train: 2 [ 200/390]  Loss: 1.132 (1.06)  Acc@1: 64.0625 (61.6527)  Acc@5: 93.7500 (96.2531)LR: 2.491e-02
Train: 2 [ 250/390]  Loss: 1.076 (1.05)  Acc@1: 54.6875 (62.0767)  Acc@5: 100.0000 (96.3894)LR: 2.491e-02
Train: 2 [ 300/390]  Loss: 0.6976 (1.04)  Acc@1: 78.1250 (62.5519)  Acc@5: 96.8750 (96.3819)LR: 2.491e-02
Train: 2 [ 350/390]  Loss: 0.8622 (1.03)  Acc@1: 70.3125 (62.7983)  Acc@5: 98.4375 (96.4922)LR: 2.491e-02
Train: 2 [ 390/390]  Loss: 0.8143 (1.02)  Acc@1: 65.0000 (63.1400)  Acc@5: 100.0000 (96.5920)LR: 2.491e-02
train_acc 63.140000
Valid: 2 [   0/390]  Loss: 0.7792 (0.779)  Acc@1: 70.3125 (70.3125)  Acc@5: 100.0000 (100.0000)
Valid: 2 [  50/390]  Loss: 0.9161 (0.986)  Acc@1: 65.6250 (65.0735)  Acc@5: 95.3125 (96.7831)
Valid: 2 [ 100/390]  Loss: 0.6962 (0.983)  Acc@1: 70.3125 (64.9288)  Acc@5: 98.4375 (96.9214)
Valid: 2 [ 150/390]  Loss: 0.8733 (0.983)  Acc@1: 73.4375 (64.8800)  Acc@5: 98.4375 (96.9681)
Valid: 2 [ 200/390]  Loss: 1.110 (0.987)  Acc@1: 64.0625 (64.8243)  Acc@5: 98.4375 (96.9216)
Valid: 2 [ 250/390]  Loss: 0.9036 (0.984)  Acc@1: 60.9375 (65.0087)  Acc@5: 98.4375 (96.9186)
Valid: 2 [ 300/390]  Loss: 1.047 (0.983)  Acc@1: 67.1875 (65.0903)  Acc@5: 93.7500 (96.8854)
Valid: 2 [ 350/390]  Loss: 0.8691 (0.980)  Acc@1: 68.7500 (65.1353)  Acc@5: 98.4375 (96.9017)
Valid: 2 [ 390/390]  Loss: 0.7659 (0.976)  Acc@1: 70.0000 (65.2640)  Acc@5: 97.5000 (96.9160)
valid_acc 65.264000
epoch = 2   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4812, 0.5188],
        [0.4778, 0.5222],
        [0.4706, 0.5294],
        [0.4756, 0.5244],
        [0.4643, 0.5357],
        [0.4777, 0.5223],
        [0.4784, 0.5216],
        [0.4699, 0.5301],
        [0.4617, 0.5383],
        [0.4702, 0.5298],
        [0.4750, 0.5250],
        [0.4606, 0.5394],
        [0.4599, 0.5401],
        [0.4542, 0.5458]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4960, 0.5040],
        [0.4884, 0.5116],
        [0.4979, 0.5021],
        [0.4749, 0.5251],
        [0.4839, 0.5161],
        [0.4762, 0.5238],
        [0.4927, 0.5073],
        [0.4836, 0.5164],
        [0.4826, 0.5174],
        [0.4836, 0.5164],
        [0.4923, 0.5077],
        [0.4828, 0.5172],
        [0.4843, 0.5157],
        [0.4696, 0.5304]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 3 [   0/390]  Loss: 0.8863 (0.886)  Acc@1: 64.0625 (64.0625)  Acc@5: 96.8750 (96.8750)LR: 2.479e-02
Train: 3 [  50/390]  Loss: 0.7375 (0.916)  Acc@1: 71.8750 (67.6471)  Acc@5: 96.8750 (97.1814)LR: 2.479e-02
Train: 3 [ 100/390]  Loss: 0.6274 (0.921)  Acc@1: 78.1250 (67.2494)  Acc@5: 100.0000 (97.3236)LR: 2.479e-02
Train: 3 [ 150/390]  Loss: 1.106 (0.916)  Acc@1: 62.5000 (67.5083)  Acc@5: 98.4375 (97.3406)LR: 2.479e-02
Train: 3 [ 200/390]  Loss: 1.019 (0.905)  Acc@1: 70.3125 (67.8949)  Acc@5: 96.8750 (97.4347)LR: 2.479e-02
Train: 3 [ 250/390]  Loss: 1.137 (0.896)  Acc@1: 64.0625 (68.3329)  Acc@5: 95.3125 (97.4913)LR: 2.479e-02
Train: 3 [ 300/390]  Loss: 0.7384 (0.893)  Acc@1: 79.6875 (68.4749)  Acc@5: 100.0000 (97.5395)LR: 2.479e-02
Train: 3 [ 350/390]  Loss: 0.9018 (0.890)  Acc@1: 67.1875 (68.6031)  Acc@5: 100.0000 (97.5383)LR: 2.479e-02
Train: 3 [ 390/390]  Loss: 0.8958 (0.889)  Acc@1: 75.0000 (68.7480)  Acc@5: 97.5000 (97.5360)LR: 2.479e-02
train_acc 68.748000
Valid: 3 [   0/390]  Loss: 0.7798 (0.780)  Acc@1: 71.8750 (71.8750)  Acc@5: 100.0000 (100.0000)
Valid: 3 [  50/390]  Loss: 1.012 (0.817)  Acc@1: 71.8750 (71.7218)  Acc@5: 95.3125 (97.7941)
Valid: 3 [ 100/390]  Loss: 0.9229 (0.833)  Acc@1: 65.6250 (70.7766)  Acc@5: 95.3125 (97.7723)
Valid: 3 [ 150/390]  Loss: 0.8893 (0.829)  Acc@1: 67.1875 (70.7575)  Acc@5: 100.0000 (97.8787)
Valid: 3 [ 200/390]  Loss: 0.7213 (0.829)  Acc@1: 76.5625 (70.8411)  Acc@5: 100.0000 (97.8700)
Valid: 3 [ 250/390]  Loss: 0.9247 (0.824)  Acc@1: 70.3125 (70.8541)  Acc@5: 92.1875 (97.9146)
Valid: 3 [ 300/390]  Loss: 0.8412 (0.825)  Acc@1: 76.5625 (70.7226)  Acc@5: 96.8750 (97.9132)
Valid: 3 [ 350/390]  Loss: 0.9495 (0.827)  Acc@1: 65.6250 (70.6642)  Acc@5: 93.7500 (97.9567)
Valid: 3 [ 390/390]  Loss: 0.8449 (0.825)  Acc@1: 72.5000 (70.7360)  Acc@5: 100.0000 (97.9360)
valid_acc 70.736000
epoch = 3   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4719, 0.5281],
        [0.4664, 0.5336],
        [0.4591, 0.5409],
        [0.4687, 0.5313],
        [0.4537, 0.5463],
        [0.4693, 0.5307],
        [0.4756, 0.5244],
        [0.4610, 0.5390],
        [0.4577, 0.5423],
        [0.4585, 0.5415],
        [0.4622, 0.5378],
        [0.4461, 0.5539],
        [0.4536, 0.5464],
        [0.4405, 0.5595]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4870, 0.5130],
        [0.4887, 0.5113],
        [0.4929, 0.5071],
        [0.4678, 0.5322],
        [0.4756, 0.5244],
        [0.4748, 0.5252],
        [0.4802, 0.5198],
        [0.4854, 0.5146],
        [0.4744, 0.5256],
        [0.4760, 0.5240],
        [0.4842, 0.5158],
        [0.4751, 0.5249],
        [0.4773, 0.5227],
        [0.4605, 0.5395]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 4 [   0/390]  Loss: 0.9998 (1.00)  Acc@1: 71.8750 (71.8750)  Acc@5: 96.8750 (96.8750)LR: 2.462e-02
Train: 4 [  50/390]  Loss: 0.8582 (0.826)  Acc@1: 68.7500 (71.2623)  Acc@5: 100.0000 (98.1311)LR: 2.462e-02
Train: 4 [ 100/390]  Loss: 0.9672 (0.816)  Acc@1: 65.6250 (71.5347)  Acc@5: 96.8750 (97.9579)LR: 2.462e-02
Train: 4 [ 150/390]  Loss: 0.6396 (0.812)  Acc@1: 76.5625 (71.5128)  Acc@5: 98.4375 (98.0650)LR: 2.462e-02
Train: 4 [ 200/390]  Loss: 0.6346 (0.813)  Acc@1: 78.1250 (71.6185)  Acc@5: 98.4375 (97.9633)LR: 2.462e-02
Train: 4 [ 250/390]  Loss: 0.5745 (0.804)  Acc@1: 81.2500 (71.8625)  Acc@5: 98.4375 (97.9955)LR: 2.462e-02
Train: 4 [ 300/390]  Loss: 0.8721 (0.803)  Acc@1: 68.7500 (71.8127)  Acc@5: 100.0000 (98.0586)LR: 2.462e-02
Train: 4 [ 350/390]  Loss: 0.9511 (0.805)  Acc@1: 62.5000 (71.7949)  Acc@5: 96.8750 (98.0280)LR: 2.462e-02
Train: 4 [ 390/390]  Loss: 0.8598 (0.802)  Acc@1: 67.5000 (71.8960)  Acc@5: 100.0000 (98.0200)LR: 2.462e-02
train_acc 71.896000
Valid: 4 [   0/390]  Loss: 0.9823 (0.982)  Acc@1: 59.3750 (59.3750)  Acc@5: 96.8750 (96.8750)
Valid: 4 [  50/390]  Loss: 0.7570 (0.888)  Acc@1: 68.7500 (68.3517)  Acc@5: 98.4375 (97.6409)
Valid: 4 [ 100/390]  Loss: 0.8259 (0.868)  Acc@1: 68.7500 (69.1832)  Acc@5: 100.0000 (97.8651)
Valid: 4 [ 150/390]  Loss: 0.9068 (0.865)  Acc@1: 65.6250 (69.0811)  Acc@5: 98.4375 (97.8580)
Valid: 4 [ 200/390]  Loss: 0.9466 (0.866)  Acc@1: 64.0625 (69.1076)  Acc@5: 98.4375 (97.9555)
Valid: 4 [ 250/390]  Loss: 0.8681 (0.868)  Acc@1: 75.0000 (69.2169)  Acc@5: 96.8750 (97.9084)
Valid: 4 [ 300/390]  Loss: 0.8987 (0.872)  Acc@1: 67.1875 (69.0303)  Acc@5: 96.8750 (97.8405)
Valid: 4 [ 350/390]  Loss: 0.8400 (0.882)  Acc@1: 75.0000 (68.8301)  Acc@5: 100.0000 (97.8054)
Valid: 4 [ 390/390]  Loss: 1.138 (0.881)  Acc@1: 57.5000 (68.8000)  Acc@5: 95.0000 (97.8280)
valid_acc 68.800000
epoch = 4   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4659, 0.5341],
        [0.4636, 0.5364],
        [0.4494, 0.5506],
        [0.4621, 0.5379],
        [0.4487, 0.5513],
        [0.4618, 0.5382],
        [0.4741, 0.5259],
        [0.4541, 0.5459],
        [0.4504, 0.5496],
        [0.4470, 0.5530],
        [0.4578, 0.5422],
        [0.4372, 0.5628],
        [0.4393, 0.5607],
        [0.4245, 0.5755]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4804, 0.5196],
        [0.4821, 0.5179],
        [0.4916, 0.5084],
        [0.4582, 0.5418],
        [0.4707, 0.5293],
        [0.4723, 0.5277],
        [0.4717, 0.5283],
        [0.4849, 0.5151],
        [0.4650, 0.5350],
        [0.4679, 0.5321],
        [0.4842, 0.5158],
        [0.4682, 0.5318],
        [0.4682, 0.5318],
        [0.4501, 0.5499]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 5 [   0/390]  Loss: 0.8711 (0.871)  Acc@1: 65.6250 (65.6250)  Acc@5: 100.0000 (100.0000)LR: 2.441e-02
Train: 5 [  50/390]  Loss: 0.7434 (0.741)  Acc@1: 76.5625 (72.9779)  Acc@5: 98.4375 (98.4988)LR: 2.441e-02
Train: 5 [ 100/390]  Loss: 0.6898 (0.728)  Acc@1: 76.5625 (73.8861)  Acc@5: 98.4375 (98.3911)LR: 2.441e-02
Train: 5 [ 150/390]  Loss: 0.8763 (0.741)  Acc@1: 73.4375 (73.7479)  Acc@5: 95.3125 (98.2926)LR: 2.441e-02
Train: 5 [ 200/390]  Loss: 0.4922 (0.741)  Acc@1: 84.3750 (73.9817)  Acc@5: 100.0000 (98.2432)LR: 2.441e-02
Train: 5 [ 250/390]  Loss: 0.8321 (0.737)  Acc@1: 67.1875 (74.0351)  Acc@5: 96.8750 (98.3130)LR: 2.441e-02
Train: 5 [ 300/390]  Loss: 0.7351 (0.732)  Acc@1: 73.4375 (74.2577)  Acc@5: 98.4375 (98.3181)LR: 2.441e-02
Train: 5 [ 350/390]  Loss: 0.7002 (0.729)  Acc@1: 78.1250 (74.4347)  Acc@5: 98.4375 (98.3351)LR: 2.441e-02
Train: 5 [ 390/390]  Loss: 0.7219 (0.726)  Acc@1: 75.0000 (74.5240)  Acc@5: 97.5000 (98.3960)LR: 2.441e-02
train_acc 74.524000
Valid: 5 [   0/390]  Loss: 0.7265 (0.726)  Acc@1: 68.7500 (68.7500)  Acc@5: 98.4375 (98.4375)
Valid: 5 [  50/390]  Loss: 1.035 (0.874)  Acc@1: 65.6250 (69.5772)  Acc@5: 93.7500 (98.2843)
Valid: 5 [ 100/390]  Loss: 0.9391 (0.864)  Acc@1: 65.6250 (69.9412)  Acc@5: 98.4375 (98.2364)
Valid: 5 [ 150/390]  Loss: 0.8336 (0.858)  Acc@1: 65.6250 (69.9503)  Acc@5: 100.0000 (98.0339)
Valid: 5 [ 200/390]  Loss: 0.9008 (0.864)  Acc@1: 60.9375 (69.8150)  Acc@5: 98.4375 (97.9944)
Valid: 5 [ 250/390]  Loss: 0.8633 (0.864)  Acc@1: 68.7500 (70.0510)  Acc@5: 98.4375 (97.9955)
Valid: 5 [ 300/390]  Loss: 0.8387 (0.862)  Acc@1: 70.3125 (70.2398)  Acc@5: 96.8750 (98.0170)
Valid: 5 [ 350/390]  Loss: 1.021 (0.861)  Acc@1: 65.6250 (70.2413)  Acc@5: 98.4375 (98.0191)
Valid: 5 [ 390/390]  Loss: 0.5548 (0.861)  Acc@1: 85.0000 (70.3800)  Acc@5: 97.5000 (97.9680)
valid_acc 70.380000
epoch = 5   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4633, 0.5367],
        [0.4534, 0.5466],
        [0.4414, 0.5586],
        [0.4543, 0.5457],
        [0.4355, 0.5645],
        [0.4483, 0.5517],
        [0.4731, 0.5269],
        [0.4443, 0.5557],
        [0.4427, 0.5573],
        [0.4301, 0.5699],
        [0.4518, 0.5482],
        [0.4218, 0.5782],
        [0.4267, 0.5733],
        [0.4057, 0.5943]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4790, 0.5210],
        [0.4737, 0.5263],
        [0.4909, 0.5091],
        [0.4520, 0.5480],
        [0.4701, 0.5299],
        [0.4673, 0.5327],
        [0.4735, 0.5265],
        [0.4876, 0.5124],
        [0.4604, 0.5396],
        [0.4601, 0.5399],
        [0.4784, 0.5216],
        [0.4640, 0.5360],
        [0.4633, 0.5367],
        [0.4450, 0.5550]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 6 [   0/390]  Loss: 0.6972 (0.697)  Acc@1: 73.4375 (73.4375)  Acc@5: 100.0000 (100.0000)LR: 2.416e-02
Train: 6 [  50/390]  Loss: 0.5191 (0.701)  Acc@1: 82.8125 (76.6544)  Acc@5: 100.0000 (98.5600)LR: 2.416e-02
Train: 6 [ 100/390]  Loss: 0.5529 (0.681)  Acc@1: 76.5625 (76.7481)  Acc@5: 100.0000 (98.5613)LR: 2.416e-02
Train: 6 [ 150/390]  Loss: 1.022 (0.681)  Acc@1: 65.6250 (76.2728)  Acc@5: 98.4375 (98.6341)LR: 2.416e-02
Train: 6 [ 200/390]  Loss: 0.4892 (0.677)  Acc@1: 84.3750 (76.2826)  Acc@5: 100.0000 (98.6629)LR: 2.416e-02
Train: 6 [ 250/390]  Loss: 0.8594 (0.680)  Acc@1: 68.7500 (76.1081)  Acc@5: 100.0000 (98.5807)LR: 2.416e-02
Train: 6 [ 300/390]  Loss: 0.5901 (0.680)  Acc@1: 81.2500 (76.1368)  Acc@5: 100.0000 (98.6192)LR: 2.416e-02
Train: 6 [ 350/390]  Loss: 0.4728 (0.681)  Acc@1: 84.3750 (76.0239)  Acc@5: 100.0000 (98.6289)LR: 2.416e-02
Train: 6 [ 390/390]  Loss: 0.6716 (0.676)  Acc@1: 72.5000 (76.2240)  Acc@5: 100.0000 (98.6200)LR: 2.416e-02
train_acc 76.224000
Valid: 6 [   0/390]  Loss: 0.8016 (0.802)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)
Valid: 6 [  50/390]  Loss: 0.6777 (0.675)  Acc@1: 71.8750 (76.5319)  Acc@5: 100.0000 (98.7745)
Valid: 6 [ 100/390]  Loss: 0.7501 (0.654)  Acc@1: 70.3125 (77.0266)  Acc@5: 100.0000 (98.6231)
Valid: 6 [ 150/390]  Loss: 0.7249 (0.659)  Acc@1: 76.5625 (77.1006)  Acc@5: 96.8750 (98.5410)
Valid: 6 [ 200/390]  Loss: 0.7737 (0.668)  Acc@1: 75.0000 (76.7491)  Acc@5: 100.0000 (98.5308)
Valid: 6 [ 250/390]  Loss: 0.8448 (0.676)  Acc@1: 68.7500 (76.4753)  Acc@5: 96.8750 (98.5122)
Valid: 6 [ 300/390]  Loss: 0.7157 (0.683)  Acc@1: 75.0000 (76.2147)  Acc@5: 100.0000 (98.4427)
Valid: 6 [ 350/390]  Loss: 0.6338 (0.683)  Acc@1: 75.0000 (76.2153)  Acc@5: 96.8750 (98.4241)
Valid: 6 [ 390/390]  Loss: 1.001 (0.685)  Acc@1: 67.5000 (76.2160)  Acc@5: 100.0000 (98.4280)
valid_acc 76.216000
epoch = 6   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4536, 0.5464],
        [0.4510, 0.5490],
        [0.4302, 0.5698],
        [0.4477, 0.5523],
        [0.4232, 0.5768],
        [0.4341, 0.5659],
        [0.4708, 0.5292],
        [0.4359, 0.5641],
        [0.4417, 0.5583],
        [0.4145, 0.5855],
        [0.4487, 0.5513],
        [0.4112, 0.5888],
        [0.4200, 0.5800],
        [0.3917, 0.6083]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4703, 0.5297],
        [0.4687, 0.5313],
        [0.4886, 0.5114],
        [0.4416, 0.5584],
        [0.4650, 0.5350],
        [0.4632, 0.5368],
        [0.4657, 0.5343],
        [0.4802, 0.5198],
        [0.4508, 0.5492],
        [0.4596, 0.5404],
        [0.4751, 0.5249],
        [0.4601, 0.5399],
        [0.4518, 0.5482],
        [0.4337, 0.5663]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 7 [   0/390]  Loss: 0.6977 (0.698)  Acc@1: 75.0000 (75.0000)  Acc@5: 96.8750 (96.8750)LR: 2.386e-02
Train: 7 [  50/390]  Loss: 0.4509 (0.628)  Acc@1: 82.8125 (78.2475)  Acc@5: 100.0000 (98.7132)LR: 2.386e-02
Train: 7 [ 100/390]  Loss: 0.6012 (0.630)  Acc@1: 82.8125 (78.2488)  Acc@5: 100.0000 (98.7469)LR: 2.386e-02
Train: 7 [ 150/390]  Loss: 0.8644 (0.638)  Acc@1: 75.0000 (78.0319)  Acc@5: 100.0000 (98.6755)LR: 2.386e-02
Train: 7 [ 200/390]  Loss: 0.6583 (0.628)  Acc@1: 78.1250 (78.3971)  Acc@5: 95.3125 (98.7251)LR: 2.386e-02
Train: 7 [ 250/390]  Loss: 0.7556 (0.629)  Acc@1: 81.2500 (78.3802)  Acc@5: 96.8750 (98.6865)LR: 2.386e-02
Train: 7 [ 300/390]  Loss: 0.6858 (0.629)  Acc@1: 75.0000 (78.1717)  Acc@5: 96.8750 (98.6503)LR: 2.386e-02
Train: 7 [ 350/390]  Loss: 0.4991 (0.626)  Acc@1: 79.6875 (78.2318)  Acc@5: 100.0000 (98.7046)LR: 2.386e-02
Train: 7 [ 390/390]  Loss: 0.3576 (0.628)  Acc@1: 87.5000 (78.1240)  Acc@5: 100.0000 (98.6560)LR: 2.386e-02
train_acc 78.124000
Valid: 7 [   0/390]  Loss: 0.6533 (0.653)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)
Valid: 7 [  50/390]  Loss: 0.8541 (0.711)  Acc@1: 68.7500 (75.2451)  Acc@5: 96.8750 (98.4069)
Valid: 7 [ 100/390]  Loss: 0.4953 (0.703)  Acc@1: 84.3750 (75.7890)  Acc@5: 100.0000 (98.3447)
Valid: 7 [ 150/390]  Loss: 0.7588 (0.706)  Acc@1: 70.3125 (75.9520)  Acc@5: 98.4375 (98.2202)
Valid: 7 [ 200/390]  Loss: 0.7479 (0.710)  Acc@1: 73.4375 (75.6919)  Acc@5: 100.0000 (98.2276)
Valid: 7 [ 250/390]  Loss: 0.8867 (0.706)  Acc@1: 73.4375 (75.8902)  Acc@5: 98.4375 (98.3068)
Valid: 7 [ 300/390]  Loss: 0.7995 (0.705)  Acc@1: 73.4375 (75.9240)  Acc@5: 98.4375 (98.2766)
Valid: 7 [ 350/390]  Loss: 0.7772 (0.703)  Acc@1: 73.4375 (75.9972)  Acc@5: 96.8750 (98.2772)
Valid: 7 [ 390/390]  Loss: 0.7505 (0.704)  Acc@1: 80.0000 (76.0000)  Acc@5: 97.5000 (98.2640)
valid_acc 76.000000
epoch = 7   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4421, 0.5579],
        [0.4452, 0.5548],
        [0.4218, 0.5782],
        [0.4372, 0.5628],
        [0.4157, 0.5843],
        [0.4268, 0.5732],
        [0.4626, 0.5374],
        [0.4285, 0.5715],
        [0.4323, 0.5677],
        [0.4013, 0.5987],
        [0.4432, 0.5568],
        [0.3994, 0.6006],
        [0.4140, 0.5860],
        [0.3770, 0.6230]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4670, 0.5330],
        [0.4603, 0.5397],
        [0.4788, 0.5212],
        [0.4334, 0.5666],
        [0.4654, 0.5346],
        [0.4595, 0.5405],
        [0.4632, 0.5368],
        [0.4784, 0.5216],
        [0.4481, 0.5519],
        [0.4547, 0.5453],
        [0.4694, 0.5306],
        [0.4563, 0.5437],
        [0.4467, 0.5533],
        [0.4249, 0.5751]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 8 [   0/390]  Loss: 0.4067 (0.407)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)LR: 2.352e-02
Train: 8 [  50/390]  Loss: 0.7062 (0.576)  Acc@1: 75.0000 (80.1164)  Acc@5: 98.4375 (98.8051)LR: 2.352e-02
Train: 8 [ 100/390]  Loss: 0.5429 (0.589)  Acc@1: 81.2500 (79.0687)  Acc@5: 98.4375 (98.7933)LR: 2.352e-02
Train: 8 [ 150/390]  Loss: 0.5991 (0.575)  Acc@1: 79.6875 (79.8634)  Acc@5: 100.0000 (98.9031)LR: 2.352e-02
Train: 8 [ 200/390]  Loss: 0.8039 (0.579)  Acc@1: 70.3125 (79.7886)  Acc@5: 96.8750 (98.9195)LR: 2.352e-02
Train: 8 [ 250/390]  Loss: 0.6798 (0.580)  Acc@1: 71.8750 (79.5879)  Acc@5: 98.4375 (98.9604)LR: 2.352e-02
Train: 8 [ 300/390]  Loss: 0.8359 (0.584)  Acc@1: 67.1875 (79.4435)  Acc@5: 96.8750 (98.9203)LR: 2.352e-02
Train: 8 [ 350/390]  Loss: 0.5335 (0.586)  Acc@1: 81.2500 (79.4293)  Acc@5: 98.4375 (98.8738)LR: 2.352e-02
Train: 8 [ 390/390]  Loss: 0.8105 (0.588)  Acc@1: 72.5000 (79.4640)  Acc@5: 95.0000 (98.8760)LR: 2.352e-02
train_acc 79.464000
Valid: 8 [   0/390]  Loss: 0.5379 (0.538)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)
Valid: 8 [  50/390]  Loss: 0.5251 (0.637)  Acc@1: 76.5625 (77.5123)  Acc@5: 100.0000 (98.6826)
Valid: 8 [ 100/390]  Loss: 0.5600 (0.670)  Acc@1: 78.1250 (76.8255)  Acc@5: 100.0000 (98.5303)
Valid: 8 [ 150/390]  Loss: 0.5772 (0.664)  Acc@1: 75.0000 (77.1109)  Acc@5: 100.0000 (98.4996)
Valid: 8 [ 200/390]  Loss: 0.6935 (0.662)  Acc@1: 78.1250 (77.2544)  Acc@5: 100.0000 (98.4764)
Valid: 8 [ 250/390]  Loss: 0.8715 (0.672)  Acc@1: 70.3125 (76.9734)  Acc@5: 98.4375 (98.4935)
Valid: 8 [ 300/390]  Loss: 0.7024 (0.673)  Acc@1: 73.4375 (76.8428)  Acc@5: 98.4375 (98.4842)
Valid: 8 [ 350/390]  Loss: 0.6706 (0.676)  Acc@1: 71.8750 (76.7450)  Acc@5: 98.4375 (98.4731)
Valid: 8 [ 390/390]  Loss: 0.6146 (0.677)  Acc@1: 80.0000 (76.7640)  Acc@5: 97.5000 (98.4680)
valid_acc 76.764000
epoch = 8   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4341, 0.5659],
        [0.4389, 0.5611],
        [0.4137, 0.5863],
        [0.4315, 0.5685],
        [0.4097, 0.5903],
        [0.4171, 0.5829],
        [0.4643, 0.5357],
        [0.4215, 0.5785],
        [0.4267, 0.5733],
        [0.3872, 0.6128],
        [0.4449, 0.5551],
        [0.3886, 0.6114],
        [0.4073, 0.5927],
        [0.3624, 0.6376]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4615, 0.5385],
        [0.4608, 0.5392],
        [0.4789, 0.5211],
        [0.4229, 0.5771],
        [0.4601, 0.5399],
        [0.4574, 0.5426],
        [0.4624, 0.5376],
        [0.4806, 0.5194],
        [0.4443, 0.5557],
        [0.4463, 0.5537],
        [0.4631, 0.5369],
        [0.4490, 0.5510],
        [0.4369, 0.5631],
        [0.4213, 0.5787]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 9 [   0/390]  Loss: 0.4708 (0.471)  Acc@1: 76.5625 (76.5625)  Acc@5: 98.4375 (98.4375)LR: 2.313e-02
Train: 9 [  50/390]  Loss: 0.4682 (0.553)  Acc@1: 79.6875 (80.3615)  Acc@5: 100.0000 (99.2034)LR: 2.313e-02
Train: 9 [ 100/390]  Loss: 0.5856 (0.559)  Acc@1: 78.1250 (80.3682)  Acc@5: 100.0000 (99.1646)LR: 2.313e-02
Train: 9 [ 150/390]  Loss: 0.8842 (0.565)  Acc@1: 60.9375 (80.0807)  Acc@5: 98.4375 (99.0170)LR: 2.313e-02
Train: 9 [ 200/390]  Loss: 0.2770 (0.563)  Acc@1: 90.6250 (80.0917)  Acc@5: 100.0000 (99.0361)LR: 2.313e-02
Train: 9 [ 250/390]  Loss: 0.4652 (0.565)  Acc@1: 85.9375 (80.1295)  Acc@5: 100.0000 (99.0040)LR: 2.313e-02
Train: 9 [ 300/390]  Loss: 0.4375 (0.557)  Acc@1: 82.8125 (80.3468)  Acc@5: 98.4375 (99.0397)LR: 2.313e-02
Train: 9 [ 350/390]  Loss: 0.4353 (0.557)  Acc@1: 82.8125 (80.4487)  Acc@5: 100.0000 (99.0741)LR: 2.313e-02
Train: 9 [ 390/390]  Loss: 0.5010 (0.554)  Acc@1: 82.5000 (80.4640)  Acc@5: 100.0000 (99.0920)LR: 2.313e-02
train_acc 80.464000
Valid: 9 [   0/390]  Loss: 0.8020 (0.802)  Acc@1: 78.1250 (78.1250)  Acc@5: 96.8750 (96.8750)
Valid: 9 [  50/390]  Loss: 0.4622 (0.628)  Acc@1: 76.5625 (78.8603)  Acc@5: 100.0000 (98.5600)
Valid: 9 [ 100/390]  Loss: 0.5021 (0.634)  Acc@1: 85.9375 (78.5736)  Acc@5: 100.0000 (98.6850)
Valid: 9 [ 150/390]  Loss: 0.8037 (0.634)  Acc@1: 78.1250 (78.5493)  Acc@5: 98.4375 (98.7169)
Valid: 9 [ 200/390]  Loss: 0.4792 (0.636)  Acc@1: 82.8125 (78.3815)  Acc@5: 95.3125 (98.6552)
Valid: 9 [ 250/390]  Loss: 0.5930 (0.638)  Acc@1: 75.0000 (78.1375)  Acc@5: 100.0000 (98.6741)
Valid: 9 [ 300/390]  Loss: 0.8545 (0.634)  Acc@1: 75.0000 (78.3482)  Acc@5: 98.4375 (98.7386)
Valid: 9 [ 350/390]  Loss: 0.6494 (0.634)  Acc@1: 82.8125 (78.4233)  Acc@5: 98.4375 (98.7224)
Valid: 9 [ 390/390]  Loss: 0.7703 (0.636)  Acc@1: 70.0000 (78.4440)  Acc@5: 97.5000 (98.7040)
valid_acc 78.444000
epoch = 9   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4265, 0.5735],
        [0.4301, 0.5699],
        [0.4048, 0.5952],
        [0.4246, 0.5754],
        [0.4085, 0.5915],
        [0.4128, 0.5872],
        [0.4632, 0.5368],
        [0.4216, 0.5784],
        [0.4246, 0.5754],
        [0.3746, 0.6254],
        [0.4392, 0.5608],
        [0.3793, 0.6207],
        [0.4005, 0.5995],
        [0.3490, 0.6510]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4538, 0.5462],
        [0.4564, 0.5436],
        [0.4777, 0.5223],
        [0.4174, 0.5826],
        [0.4545, 0.5455],
        [0.4519, 0.5481],
        [0.4567, 0.5433],
        [0.4763, 0.5237],
        [0.4350, 0.5650],
        [0.4402, 0.5598],
        [0.4564, 0.5436],
        [0.4405, 0.5595],
        [0.4262, 0.5738],
        [0.4135, 0.5865]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 10 [   0/390]  Loss: 0.3882 (0.388)  Acc@1: 84.3750 (84.3750)  Acc@5: 98.4375 (98.4375)LR: 2.271e-02
Train: 10 [  50/390]  Loss: 0.5707 (0.518)  Acc@1: 78.1250 (80.9743)  Acc@5: 100.0000 (99.0809)LR: 2.271e-02
Train: 10 [ 100/390]  Loss: 0.4667 (0.523)  Acc@1: 79.6875 (81.0953)  Acc@5: 98.4375 (99.0718)LR: 2.271e-02
Train: 10 [ 150/390]  Loss: 0.4645 (0.536)  Acc@1: 81.2500 (81.1672)  Acc@5: 100.0000 (98.9549)LR: 2.271e-02
Train: 10 [ 200/390]  Loss: 0.5151 (0.534)  Acc@1: 82.8125 (81.4443)  Acc@5: 98.4375 (98.9428)LR: 2.271e-02
Train: 10 [ 250/390]  Loss: 0.4230 (0.532)  Acc@1: 82.8125 (81.4990)  Acc@5: 100.0000 (98.9666)LR: 2.271e-02
Train: 10 [ 300/390]  Loss: 0.7080 (0.528)  Acc@1: 79.6875 (81.6030)  Acc@5: 98.4375 (99.0397)LR: 2.271e-02
Train: 10 [ 350/390]  Loss: 0.3707 (0.527)  Acc@1: 90.6250 (81.6907)  Acc@5: 100.0000 (99.0741)LR: 2.271e-02
Train: 10 [ 390/390]  Loss: 0.6525 (0.526)  Acc@1: 82.5000 (81.6280)  Acc@5: 100.0000 (99.0680)LR: 2.271e-02
train_acc 81.628000
Valid: 10 [   0/390]  Loss: 0.3003 (0.300)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 10 [  50/390]  Loss: 0.5257 (0.546)  Acc@1: 71.8750 (81.6789)  Acc@5: 100.0000 (98.8971)
Valid: 10 [ 100/390]  Loss: 0.4933 (0.554)  Acc@1: 84.3750 (81.0644)  Acc@5: 98.4375 (98.9635)
Valid: 10 [ 150/390]  Loss: 0.4671 (0.552)  Acc@1: 85.9375 (81.3742)  Acc@5: 100.0000 (99.0480)
Valid: 10 [ 200/390]  Loss: 0.5483 (0.552)  Acc@1: 82.8125 (81.4132)  Acc@5: 98.4375 (99.0672)
Valid: 10 [ 250/390]  Loss: 0.4360 (0.554)  Acc@1: 87.5000 (81.3870)  Acc@5: 100.0000 (99.0725)
Valid: 10 [ 300/390]  Loss: 0.5349 (0.560)  Acc@1: 79.6875 (81.1462)  Acc@5: 100.0000 (99.0397)
Valid: 10 [ 350/390]  Loss: 0.4386 (0.565)  Acc@1: 82.8125 (80.9606)  Acc@5: 100.0000 (99.0429)
Valid: 10 [ 390/390]  Loss: 0.6874 (0.567)  Acc@1: 85.0000 (80.9720)  Acc@5: 97.5000 (99.0040)
valid_acc 80.972000
epoch = 10   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4226, 0.5774],
        [0.4250, 0.5750],
        [0.3976, 0.6024],
        [0.4202, 0.5798],
        [0.4065, 0.5935],
        [0.4094, 0.5906],
        [0.4568, 0.5432],
        [0.4233, 0.5767],
        [0.4179, 0.5821],
        [0.3662, 0.6338],
        [0.4415, 0.5585],
        [0.3723, 0.6277],
        [0.3950, 0.6050],
        [0.3372, 0.6628]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4470, 0.5530],
        [0.4531, 0.5469],
        [0.4808, 0.5192],
        [0.4085, 0.5915],
        [0.4531, 0.5469],
        [0.4440, 0.5560],
        [0.4502, 0.5498],
        [0.4689, 0.5311],
        [0.4195, 0.5805],
        [0.4399, 0.5601],
        [0.4496, 0.5504],
        [0.4370, 0.5630],
        [0.4146, 0.5854],
        [0.4035, 0.5965]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 11 [   0/390]  Loss: 0.4545 (0.455)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 2.225e-02
Train: 11 [  50/390]  Loss: 0.4688 (0.481)  Acc@1: 81.2500 (83.1189)  Acc@5: 100.0000 (99.2953)LR: 2.225e-02
Train: 11 [ 100/390]  Loss: 0.5098 (0.474)  Acc@1: 84.3750 (83.4004)  Acc@5: 100.0000 (99.3038)LR: 2.225e-02
Train: 11 [ 150/390]  Loss: 0.4649 (0.485)  Acc@1: 82.8125 (83.2368)  Acc@5: 98.4375 (99.2757)LR: 2.225e-02
Train: 11 [ 200/390]  Loss: 0.5649 (0.499)  Acc@1: 79.6875 (82.7348)  Acc@5: 98.4375 (99.2149)LR: 2.225e-02
Train: 11 [ 250/390]  Loss: 0.6039 (0.496)  Acc@1: 81.2500 (82.8374)  Acc@5: 100.0000 (99.2468)LR: 2.225e-02
Train: 11 [ 300/390]  Loss: 0.6793 (0.495)  Acc@1: 76.5625 (82.9007)  Acc@5: 98.4375 (99.2473)LR: 2.225e-02
Train: 11 [ 350/390]  Loss: 0.4650 (0.497)  Acc@1: 87.5000 (82.8437)  Acc@5: 100.0000 (99.2076)LR: 2.225e-02
Train: 11 [ 390/390]  Loss: 0.3724 (0.499)  Acc@1: 82.5000 (82.7440)  Acc@5: 100.0000 (99.2080)LR: 2.225e-02
train_acc 82.744000
Valid: 11 [   0/390]  Loss: 0.3593 (0.359)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 11 [  50/390]  Loss: 0.5471 (0.566)  Acc@1: 81.2500 (81.1275)  Acc@5: 100.0000 (98.8971)
Valid: 11 [ 100/390]  Loss: 0.8014 (0.586)  Acc@1: 71.8750 (80.3527)  Acc@5: 96.8750 (98.7933)
Valid: 11 [ 150/390]  Loss: 0.8839 (0.587)  Acc@1: 70.3125 (80.0704)  Acc@5: 95.3125 (98.8100)
Valid: 11 [ 200/390]  Loss: 0.5092 (0.584)  Acc@1: 82.8125 (80.2083)  Acc@5: 100.0000 (98.8029)
Valid: 11 [ 250/390]  Loss: 0.5559 (0.592)  Acc@1: 81.2500 (79.9925)  Acc@5: 100.0000 (98.7425)
Valid: 11 [ 300/390]  Loss: 0.6842 (0.589)  Acc@1: 70.3125 (80.1391)  Acc@5: 98.4375 (98.7957)
Valid: 11 [ 350/390]  Loss: 0.5974 (0.593)  Acc@1: 78.1250 (80.0481)  Acc@5: 98.4375 (98.7758)
Valid: 11 [ 390/390]  Loss: 0.6972 (0.589)  Acc@1: 77.5000 (80.1320)  Acc@5: 100.0000 (98.7840)
valid_acc 80.132000
epoch = 11   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4135, 0.5865],
        [0.4186, 0.5814],
        [0.3864, 0.6136],
        [0.4083, 0.5917],
        [0.4031, 0.5969],
        [0.4020, 0.5980],
        [0.4498, 0.5502],
        [0.4217, 0.5783],
        [0.4129, 0.5871],
        [0.3550, 0.6450],
        [0.4461, 0.5539],
        [0.3640, 0.6360],
        [0.3903, 0.6097],
        [0.3247, 0.6753]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4401, 0.5599],
        [0.4485, 0.5515],
        [0.4784, 0.5216],
        [0.4003, 0.5997],
        [0.4521, 0.5479],
        [0.4403, 0.5597],
        [0.4452, 0.5548],
        [0.4621, 0.5379],
        [0.4080, 0.5920],
        [0.4322, 0.5678],
        [0.4476, 0.5524],
        [0.4383, 0.5617],
        [0.4069, 0.5931],
        [0.4082, 0.5918]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 12 [   0/390]  Loss: 0.2881 (0.288)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 2.175e-02
Train: 12 [  50/390]  Loss: 0.4047 (0.461)  Acc@1: 82.8125 (83.7010)  Acc@5: 100.0000 (99.3566)LR: 2.175e-02
Train: 12 [ 100/390]  Loss: 0.4897 (0.470)  Acc@1: 82.8125 (83.8800)  Acc@5: 98.4375 (99.3502)LR: 2.175e-02
Train: 12 [ 150/390]  Loss: 0.7752 (0.469)  Acc@1: 73.4375 (83.8887)  Acc@5: 98.4375 (99.3584)LR: 2.175e-02
Train: 12 [ 200/390]  Loss: 0.4019 (0.470)  Acc@1: 82.8125 (83.8308)  Acc@5: 100.0000 (99.2693)LR: 2.175e-02
Train: 12 [ 250/390]  Loss: 0.3743 (0.472)  Acc@1: 89.0625 (83.6529)  Acc@5: 100.0000 (99.2779)LR: 2.175e-02
Train: 12 [ 300/390]  Loss: 0.4288 (0.478)  Acc@1: 82.8125 (83.3887)  Acc@5: 100.0000 (99.3096)LR: 2.175e-02
Train: 12 [ 350/390]  Loss: 0.5904 (0.482)  Acc@1: 81.2500 (83.2621)  Acc@5: 100.0000 (99.2922)LR: 2.175e-02
Train: 12 [ 390/390]  Loss: 0.4237 (0.479)  Acc@1: 77.5000 (83.2880)  Acc@5: 100.0000 (99.3120)LR: 2.175e-02
train_acc 83.288000
Valid: 12 [   0/390]  Loss: 0.5610 (0.561)  Acc@1: 82.8125 (82.8125)  Acc@5: 96.8750 (96.8750)
Valid: 12 [  50/390]  Loss: 0.5892 (0.558)  Acc@1: 78.1250 (81.7402)  Acc@5: 98.4375 (99.2953)
Valid: 12 [ 100/390]  Loss: 0.7160 (0.564)  Acc@1: 78.1250 (81.5594)  Acc@5: 100.0000 (99.1491)
Valid: 12 [ 150/390]  Loss: 0.7921 (0.556)  Acc@1: 81.2500 (81.3121)  Acc@5: 96.8750 (99.0687)
Valid: 12 [ 200/390]  Loss: 0.4201 (0.555)  Acc@1: 85.9375 (81.1412)  Acc@5: 98.4375 (99.1371)
Valid: 12 [ 250/390]  Loss: 0.5721 (0.558)  Acc@1: 78.1250 (81.1566)  Acc@5: 98.4375 (98.9915)
Valid: 12 [ 300/390]  Loss: 0.3880 (0.551)  Acc@1: 85.9375 (81.3694)  Acc@5: 98.4375 (99.0293)
Valid: 12 [ 350/390]  Loss: 0.4706 (0.550)  Acc@1: 81.2500 (81.4503)  Acc@5: 100.0000 (99.0251)
Valid: 12 [ 390/390]  Loss: 0.4085 (0.552)  Acc@1: 77.5000 (81.3920)  Acc@5: 100.0000 (99.0080)
valid_acc 81.392000
epoch = 12   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4069, 0.5931],
        [0.4201, 0.5799],
        [0.3777, 0.6223],
        [0.4010, 0.5990],
        [0.3978, 0.6022],
        [0.3983, 0.6017],
        [0.4460, 0.5540],
        [0.4214, 0.5786],
        [0.4120, 0.5880],
        [0.3422, 0.6578],
        [0.4457, 0.5543],
        [0.3573, 0.6427],
        [0.3837, 0.6163],
        [0.3140, 0.6860]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4356, 0.5644],
        [0.4486, 0.5514],
        [0.4778, 0.5222],
        [0.3910, 0.6090],
        [0.4489, 0.5511],
        [0.4392, 0.5608],
        [0.4454, 0.5546],
        [0.4591, 0.5409],
        [0.4074, 0.5926],
        [0.4228, 0.5772],
        [0.4390, 0.5610],
        [0.4351, 0.5649],
        [0.4012, 0.5988],
        [0.4109, 0.5891]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 13 [   0/390]  Loss: 0.4432 (0.443)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)LR: 2.121e-02
Train: 13 [  50/390]  Loss: 0.5765 (0.450)  Acc@1: 78.1250 (84.0686)  Acc@5: 100.0000 (99.4792)LR: 2.121e-02
Train: 13 [ 100/390]  Loss: 0.3562 (0.453)  Acc@1: 87.5000 (84.0965)  Acc@5: 98.4375 (99.3657)LR: 2.121e-02
Train: 13 [ 150/390]  Loss: 0.4617 (0.458)  Acc@1: 85.9375 (84.0749)  Acc@5: 100.0000 (99.3791)LR: 2.121e-02
Train: 13 [ 200/390]  Loss: 0.2537 (0.459)  Acc@1: 92.1875 (84.1340)  Acc@5: 100.0000 (99.3703)LR: 2.121e-02
Train: 13 [ 250/390]  Loss: 0.3770 (0.462)  Acc@1: 84.3750 (83.9268)  Acc@5: 100.0000 (99.3899)LR: 2.121e-02
Train: 13 [ 300/390]  Loss: 0.4711 (0.463)  Acc@1: 79.6875 (83.8767)  Acc@5: 98.4375 (99.3771)LR: 2.121e-02
Train: 13 [ 350/390]  Loss: 0.6359 (0.461)  Acc@1: 75.0000 (84.0589)  Acc@5: 100.0000 (99.3590)LR: 2.121e-02
Train: 13 [ 390/390]  Loss: 0.5819 (0.466)  Acc@1: 75.0000 (83.8120)  Acc@5: 100.0000 (99.3760)LR: 2.121e-02
train_acc 83.812000
Valid: 13 [   0/390]  Loss: 0.5028 (0.503)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 13 [  50/390]  Loss: 0.4718 (0.621)  Acc@1: 84.3750 (79.7488)  Acc@5: 100.0000 (98.5907)
Valid: 13 [ 100/390]  Loss: 0.5779 (0.615)  Acc@1: 81.2500 (79.5173)  Acc@5: 100.0000 (98.7469)
Valid: 13 [ 150/390]  Loss: 0.6589 (0.618)  Acc@1: 78.1250 (79.4702)  Acc@5: 96.8750 (98.6755)
Valid: 13 [ 200/390]  Loss: 0.4950 (0.612)  Acc@1: 85.9375 (79.7419)  Acc@5: 100.0000 (98.7096)
Valid: 13 [ 250/390]  Loss: 0.7933 (0.616)  Acc@1: 73.4375 (79.6003)  Acc@5: 98.4375 (98.6803)
Valid: 13 [ 300/390]  Loss: 0.3601 (0.617)  Acc@1: 90.6250 (79.5941)  Acc@5: 98.4375 (98.5621)
Valid: 13 [ 350/390]  Loss: 0.6518 (0.620)  Acc@1: 79.6875 (79.6341)  Acc@5: 100.0000 (98.5532)
Valid: 13 [ 390/390]  Loss: 0.5100 (0.623)  Acc@1: 77.5000 (79.6200)  Acc@5: 100.0000 (98.5160)
valid_acc 79.620000
epoch = 13   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3955, 0.6045],
        [0.4074, 0.5926],
        [0.3678, 0.6322],
        [0.3905, 0.6095],
        [0.3974, 0.6026],
        [0.3990, 0.6010],
        [0.4444, 0.5556],
        [0.4212, 0.5788],
        [0.4116, 0.5884],
        [0.3327, 0.6673],
        [0.4462, 0.5538],
        [0.3530, 0.6470],
        [0.3788, 0.6212],
        [0.2985, 0.7015]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4326, 0.5674],
        [0.4422, 0.5578],
        [0.4769, 0.5231],
        [0.3859, 0.6141],
        [0.4455, 0.5545],
        [0.4331, 0.5669],
        [0.4450, 0.5550],
        [0.4493, 0.5507],
        [0.3988, 0.6012],
        [0.4149, 0.5851],
        [0.4343, 0.5657],
        [0.4285, 0.5715],
        [0.3964, 0.6036],
        [0.4086, 0.5914]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 14 [   0/390]  Loss: 0.3947 (0.395)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)LR: 2.065e-02
Train: 14 [  50/390]  Loss: 0.3069 (0.451)  Acc@1: 90.6250 (84.5588)  Acc@5: 100.0000 (99.3566)LR: 2.065e-02
Train: 14 [ 100/390]  Loss: 0.6947 (0.433)  Acc@1: 78.1250 (85.0248)  Acc@5: 100.0000 (99.3967)LR: 2.065e-02
Train: 14 [ 150/390]  Loss: 0.3373 (0.437)  Acc@1: 87.5000 (84.7889)  Acc@5: 100.0000 (99.3688)LR: 2.065e-02
Train: 14 [ 200/390]  Loss: 0.4239 (0.446)  Acc@1: 84.3750 (84.5460)  Acc@5: 100.0000 (99.3703)LR: 2.065e-02
Train: 14 [ 250/390]  Loss: 0.3707 (0.440)  Acc@1: 84.3750 (84.8481)  Acc@5: 98.4375 (99.3837)LR: 2.065e-02
Train: 14 [ 300/390]  Loss: 0.4487 (0.445)  Acc@1: 85.9375 (84.6242)  Acc@5: 100.0000 (99.3771)LR: 2.065e-02
Train: 14 [ 350/390]  Loss: 0.5740 (0.440)  Acc@1: 84.3750 (84.7934)  Acc@5: 100.0000 (99.4168)LR: 2.065e-02
Train: 14 [ 390/390]  Loss: 0.6756 (0.442)  Acc@1: 77.5000 (84.6640)  Acc@5: 100.0000 (99.4080)LR: 2.065e-02
train_acc 84.664000
Valid: 14 [   0/390]  Loss: 0.5178 (0.518)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 14 [  50/390]  Loss: 0.6962 (0.609)  Acc@1: 78.1250 (79.1973)  Acc@5: 96.8750 (98.6520)
Valid: 14 [ 100/390]  Loss: 0.6321 (0.603)  Acc@1: 78.1250 (79.3317)  Acc@5: 100.0000 (98.8861)
Valid: 14 [ 150/390]  Loss: 0.6704 (0.597)  Acc@1: 78.1250 (79.7185)  Acc@5: 95.3125 (98.8514)
Valid: 14 [ 200/390]  Loss: 0.4848 (0.587)  Acc@1: 79.6875 (79.9751)  Acc@5: 100.0000 (98.8806)
Valid: 14 [ 250/390]  Loss: 0.6471 (0.582)  Acc@1: 84.3750 (80.2166)  Acc@5: 98.4375 (98.9542)
Valid: 14 [ 300/390]  Loss: 0.6507 (0.581)  Acc@1: 75.0000 (80.3104)  Acc@5: 98.4375 (98.9462)
Valid: 14 [ 350/390]  Loss: 0.4024 (0.577)  Acc@1: 82.8125 (80.3819)  Acc@5: 100.0000 (98.9494)
Valid: 14 [ 390/390]  Loss: 0.6641 (0.580)  Acc@1: 80.0000 (80.3880)  Acc@5: 97.5000 (98.9400)
valid_acc 80.388000
epoch = 14   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3854, 0.6146],
        [0.4020, 0.5980],
        [0.3588, 0.6412],
        [0.3793, 0.6207],
        [0.3866, 0.6134],
        [0.3975, 0.6025],
        [0.4428, 0.5572],
        [0.4162, 0.5838],
        [0.4061, 0.5939],
        [0.3252, 0.6748],
        [0.4405, 0.5595],
        [0.3467, 0.6533],
        [0.3718, 0.6282],
        [0.2846, 0.7154]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4294, 0.5706],
        [0.4375, 0.5625],
        [0.4752, 0.5248],
        [0.3815, 0.6185],
        [0.4446, 0.5554],
        [0.4240, 0.5760],
        [0.4384, 0.5616],
        [0.4415, 0.5585],
        [0.3933, 0.6067],
        [0.4105, 0.5895],
        [0.4309, 0.5691],
        [0.4257, 0.5743],
        [0.3901, 0.6099],
        [0.4054, 0.5946]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 15 [   0/390]  Loss: 0.3771 (0.377)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 2.005e-02
Train: 15 [  50/390]  Loss: 0.3682 (0.421)  Acc@1: 87.5000 (85.0184)  Acc@5: 100.0000 (99.6936)LR: 2.005e-02
Train: 15 [ 100/390]  Loss: 0.3377 (0.416)  Acc@1: 87.5000 (85.3806)  Acc@5: 100.0000 (99.5668)LR: 2.005e-02
Train: 15 [ 150/390]  Loss: 0.4738 (0.413)  Acc@1: 78.1250 (85.4098)  Acc@5: 100.0000 (99.5964)LR: 2.005e-02
Train: 15 [ 200/390]  Loss: 0.3950 (0.416)  Acc@1: 79.6875 (85.2456)  Acc@5: 100.0000 (99.5647)LR: 2.005e-02
Train: 15 [ 250/390]  Loss: 0.5990 (0.419)  Acc@1: 81.2500 (85.3150)  Acc@5: 98.4375 (99.5331)LR: 2.005e-02
Train: 15 [ 300/390]  Loss: 0.4503 (0.428)  Acc@1: 81.2500 (85.1017)  Acc@5: 98.4375 (99.4913)LR: 2.005e-02
Train: 15 [ 350/390]  Loss: 0.5289 (0.426)  Acc@1: 76.5625 (85.0694)  Acc@5: 100.0000 (99.5148)LR: 2.005e-02
Train: 15 [ 390/390]  Loss: 0.4296 (0.426)  Acc@1: 87.5000 (85.1400)  Acc@5: 97.5000 (99.5240)LR: 2.005e-02
train_acc 85.140000
Valid: 15 [   0/390]  Loss: 0.2501 (0.250)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 15 [  50/390]  Loss: 0.8033 (0.500)  Acc@1: 76.5625 (83.3333)  Acc@5: 96.8750 (99.1115)
Valid: 15 [ 100/390]  Loss: 0.6277 (0.515)  Acc@1: 78.1250 (83.0446)  Acc@5: 100.0000 (99.1182)
Valid: 15 [ 150/390]  Loss: 0.4489 (0.521)  Acc@1: 82.8125 (82.6469)  Acc@5: 98.4375 (99.0894)
Valid: 15 [ 200/390]  Loss: 0.3211 (0.530)  Acc@1: 87.5000 (82.2683)  Acc@5: 100.0000 (99.0905)
Valid: 15 [ 250/390]  Loss: 0.4080 (0.520)  Acc@1: 87.5000 (82.5822)  Acc@5: 100.0000 (99.1285)
Valid: 15 [ 300/390]  Loss: 0.5650 (0.524)  Acc@1: 82.8125 (82.5218)  Acc@5: 100.0000 (99.1487)
Valid: 15 [ 350/390]  Loss: 0.5251 (0.523)  Acc@1: 82.8125 (82.5321)  Acc@5: 98.4375 (99.1631)
Valid: 15 [ 390/390]  Loss: 0.3286 (0.525)  Acc@1: 85.0000 (82.3920)  Acc@5: 100.0000 (99.1400)
valid_acc 82.392000
epoch = 15   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3723, 0.6277],
        [0.3937, 0.6063],
        [0.3442, 0.6558],
        [0.3730, 0.6270],
        [0.3832, 0.6168],
        [0.3906, 0.6094],
        [0.4354, 0.5646],
        [0.4208, 0.5792],
        [0.4021, 0.5979],
        [0.3149, 0.6851],
        [0.4387, 0.5613],
        [0.3428, 0.6572],
        [0.3711, 0.6289],
        [0.2729, 0.7271]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4279, 0.5721],
        [0.4373, 0.5627],
        [0.4736, 0.5264],
        [0.3783, 0.6217],
        [0.4375, 0.5625],
        [0.4233, 0.5767],
        [0.4344, 0.5656],
        [0.4339, 0.5661],
        [0.3896, 0.6104],
        [0.4066, 0.5934],
        [0.4287, 0.5713],
        [0.4218, 0.5782],
        [0.3871, 0.6129],
        [0.4062, 0.5938]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 16 [   0/390]  Loss: 0.5273 (0.527)  Acc@1: 75.0000 (75.0000)  Acc@5: 100.0000 (100.0000)LR: 1.943e-02
Train: 16 [  50/390]  Loss: 0.3565 (0.395)  Acc@1: 92.1875 (86.5502)  Acc@5: 100.0000 (99.3873)LR: 1.943e-02
Train: 16 [ 100/390]  Loss: 0.4067 (0.378)  Acc@1: 84.3750 (87.0359)  Acc@5: 100.0000 (99.5359)LR: 1.943e-02
Train: 16 [ 150/390]  Loss: 0.3542 (0.385)  Acc@1: 85.9375 (86.4963)  Acc@5: 100.0000 (99.5550)LR: 1.943e-02
Train: 16 [ 200/390]  Loss: 0.3346 (0.390)  Acc@1: 90.6250 (86.4739)  Acc@5: 100.0000 (99.5336)LR: 1.943e-02
Train: 16 [ 250/390]  Loss: 0.5118 (0.391)  Acc@1: 84.3750 (86.3919)  Acc@5: 98.4375 (99.5207)LR: 1.943e-02
Train: 16 [ 300/390]  Loss: 0.3942 (0.398)  Acc@1: 81.2500 (86.0725)  Acc@5: 100.0000 (99.5224)LR: 1.943e-02
Train: 16 [ 350/390]  Loss: 0.5399 (0.400)  Acc@1: 84.3750 (86.0399)  Acc@5: 98.4375 (99.4925)LR: 1.943e-02
Train: 16 [ 390/390]  Loss: 0.4515 (0.403)  Acc@1: 85.0000 (85.9600)  Acc@5: 100.0000 (99.5040)LR: 1.943e-02
train_acc 85.960000
Valid: 16 [   0/390]  Loss: 0.6703 (0.670)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)
Valid: 16 [  50/390]  Loss: 0.6898 (0.525)  Acc@1: 76.5625 (81.7402)  Acc@5: 96.8750 (99.2034)
Valid: 16 [ 100/390]  Loss: 0.6922 (0.498)  Acc@1: 76.5625 (82.9827)  Acc@5: 98.4375 (99.1801)
Valid: 16 [ 150/390]  Loss: 0.3763 (0.495)  Acc@1: 90.6250 (83.1229)  Acc@5: 100.0000 (99.2446)
Valid: 16 [ 200/390]  Loss: 0.5979 (0.496)  Acc@1: 79.6875 (83.1234)  Acc@5: 96.8750 (99.2149)
Valid: 16 [ 250/390]  Loss: 0.4607 (0.505)  Acc@1: 84.3750 (82.8685)  Acc@5: 100.0000 (99.1098)
Valid: 16 [ 300/390]  Loss: 0.4044 (0.502)  Acc@1: 82.8125 (82.9734)  Acc@5: 100.0000 (99.1279)
Valid: 16 [ 350/390]  Loss: 0.3806 (0.500)  Acc@1: 84.3750 (83.0440)  Acc@5: 100.0000 (99.1052)
Valid: 16 [ 390/390]  Loss: 0.4698 (0.501)  Acc@1: 72.5000 (82.9240)  Acc@5: 100.0000 (99.0840)
valid_acc 82.924000
epoch = 16   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3616, 0.6384],
        [0.3862, 0.6138],
        [0.3346, 0.6654],
        [0.3660, 0.6340],
        [0.3776, 0.6224],
        [0.3825, 0.6175],
        [0.4316, 0.5684],
        [0.4169, 0.5831],
        [0.3976, 0.6024],
        [0.3081, 0.6919],
        [0.4375, 0.5625],
        [0.3386, 0.6614],
        [0.3722, 0.6278],
        [0.2625, 0.7375]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4266, 0.5734],
        [0.4338, 0.5662],
        [0.4682, 0.5318],
        [0.3689, 0.6311],
        [0.4343, 0.5657],
        [0.4154, 0.5846],
        [0.4304, 0.5696],
        [0.4267, 0.5733],
        [0.3816, 0.6184],
        [0.4036, 0.5964],
        [0.4289, 0.5711],
        [0.4197, 0.5803],
        [0.3802, 0.6198],
        [0.4040, 0.5960]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 17 [   0/390]  Loss: 0.3283 (0.328)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.878e-02
Train: 17 [  50/390]  Loss: 0.3208 (0.359)  Acc@1: 93.7500 (87.0711)  Acc@5: 100.0000 (99.7243)LR: 1.878e-02
Train: 17 [ 100/390]  Loss: 0.5558 (0.375)  Acc@1: 85.9375 (86.6801)  Acc@5: 100.0000 (99.6442)LR: 1.878e-02
Train: 17 [ 150/390]  Loss: 0.3276 (0.382)  Acc@1: 85.9375 (86.4342)  Acc@5: 100.0000 (99.5964)LR: 1.878e-02
Train: 17 [ 200/390]  Loss: 0.3239 (0.388)  Acc@1: 84.3750 (86.2329)  Acc@5: 100.0000 (99.5491)LR: 1.878e-02
Train: 17 [ 250/390]  Loss: 0.2839 (0.391)  Acc@1: 90.6250 (86.1492)  Acc@5: 100.0000 (99.5331)LR: 1.878e-02
Train: 17 [ 300/390]  Loss: 0.4223 (0.390)  Acc@1: 89.0625 (86.1244)  Acc@5: 100.0000 (99.4965)LR: 1.878e-02
Train: 17 [ 350/390]  Loss: 0.3426 (0.386)  Acc@1: 87.5000 (86.2625)  Acc@5: 100.0000 (99.5103)LR: 1.878e-02
Train: 17 [ 390/390]  Loss: 0.4346 (0.387)  Acc@1: 82.5000 (86.2040)  Acc@5: 100.0000 (99.5440)LR: 1.878e-02
train_acc 86.204000
Valid: 17 [   0/390]  Loss: 0.2922 (0.292)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 17 [  50/390]  Loss: 0.4946 (0.491)  Acc@1: 82.8125 (83.6091)  Acc@5: 98.4375 (99.2034)
Valid: 17 [ 100/390]  Loss: 0.3176 (0.524)  Acc@1: 89.0625 (82.4876)  Acc@5: 100.0000 (99.1337)
Valid: 17 [ 150/390]  Loss: 0.4724 (0.523)  Acc@1: 84.3750 (82.7090)  Acc@5: 98.4375 (99.0066)
Valid: 17 [ 200/390]  Loss: 0.4122 (0.514)  Acc@1: 85.9375 (82.8980)  Acc@5: 98.4375 (99.0672)
Valid: 17 [ 250/390]  Loss: 0.4512 (0.515)  Acc@1: 82.8125 (82.8872)  Acc@5: 100.0000 (99.0413)
Valid: 17 [ 300/390]  Loss: 0.4379 (0.513)  Acc@1: 84.3750 (82.9059)  Acc@5: 98.4375 (99.0552)
Valid: 17 [ 350/390]  Loss: 0.5071 (0.512)  Acc@1: 84.3750 (82.9683)  Acc@5: 100.0000 (99.0340)
Valid: 17 [ 390/390]  Loss: 0.7549 (0.510)  Acc@1: 82.5000 (82.9320)  Acc@5: 97.5000 (99.0440)
valid_acc 82.932000
epoch = 17   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3549, 0.6451],
        [0.3834, 0.6166],
        [0.3259, 0.6741],
        [0.3630, 0.6370],
        [0.3722, 0.6278],
        [0.3794, 0.6206],
        [0.4286, 0.5714],
        [0.4152, 0.5848],
        [0.3914, 0.6086],
        [0.3024, 0.6976],
        [0.4351, 0.5649],
        [0.3373, 0.6627],
        [0.3683, 0.6317],
        [0.2512, 0.7488]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4241, 0.5759],
        [0.4271, 0.5729],
        [0.4658, 0.5342],
        [0.3629, 0.6371],
        [0.4324, 0.5676],
        [0.4077, 0.5923],
        [0.4307, 0.5693],
        [0.4235, 0.5765],
        [0.3781, 0.6219],
        [0.3989, 0.6011],
        [0.4260, 0.5740],
        [0.4137, 0.5863],
        [0.3752, 0.6248],
        [0.4008, 0.5992]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 18 [   0/390]  Loss: 0.2336 (0.234)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 1.811e-02
Train: 18 [  50/390]  Loss: 0.2881 (0.340)  Acc@1: 89.0625 (88.1740)  Acc@5: 100.0000 (99.5404)LR: 1.811e-02
Train: 18 [ 100/390]  Loss: 0.3518 (0.358)  Acc@1: 92.1875 (87.3917)  Acc@5: 100.0000 (99.5978)LR: 1.811e-02
Train: 18 [ 150/390]  Loss: 0.2685 (0.362)  Acc@1: 87.5000 (87.1378)  Acc@5: 100.0000 (99.6068)LR: 1.811e-02
Train: 18 [ 200/390]  Loss: 0.2792 (0.362)  Acc@1: 89.0625 (87.1113)  Acc@5: 100.0000 (99.6424)LR: 1.811e-02
Train: 18 [ 250/390]  Loss: 0.4414 (0.360)  Acc@1: 85.9375 (87.1825)  Acc@5: 100.0000 (99.6576)LR: 1.811e-02
Train: 18 [ 300/390]  Loss: 0.5391 (0.365)  Acc@1: 78.1250 (87.0432)  Acc@5: 100.0000 (99.6211)LR: 1.811e-02
Train: 18 [ 350/390]  Loss: 0.2030 (0.369)  Acc@1: 96.8750 (86.9436)  Acc@5: 100.0000 (99.6216)LR: 1.811e-02
Train: 18 [ 390/390]  Loss: 0.5121 (0.372)  Acc@1: 77.5000 (86.8120)  Acc@5: 100.0000 (99.6120)LR: 1.811e-02
train_acc 86.812000
Valid: 18 [   0/390]  Loss: 0.3608 (0.361)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 18 [  50/390]  Loss: 0.5730 (0.482)  Acc@1: 75.0000 (84.0993)  Acc@5: 100.0000 (98.9890)
Valid: 18 [ 100/390]  Loss: 0.6009 (0.502)  Acc@1: 78.1250 (83.2921)  Acc@5: 98.4375 (98.9635)
Valid: 18 [ 150/390]  Loss: 0.2980 (0.500)  Acc@1: 89.0625 (83.2161)  Acc@5: 100.0000 (99.0480)
Valid: 18 [ 200/390]  Loss: 0.4215 (0.501)  Acc@1: 84.3750 (83.1701)  Acc@5: 100.0000 (99.0361)
Valid: 18 [ 250/390]  Loss: 0.4378 (0.498)  Acc@1: 82.8125 (83.3292)  Acc@5: 100.0000 (99.0413)
Valid: 18 [ 300/390]  Loss: 0.4068 (0.502)  Acc@1: 85.9375 (83.0617)  Acc@5: 98.4375 (98.9774)
Valid: 18 [ 350/390]  Loss: 0.5443 (0.500)  Acc@1: 82.8125 (83.1775)  Acc@5: 100.0000 (99.0073)
Valid: 18 [ 390/390]  Loss: 0.8045 (0.499)  Acc@1: 70.0000 (83.1560)  Acc@5: 97.5000 (99.0160)
valid_acc 83.156000
epoch = 18   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3474, 0.6526],
        [0.3792, 0.6208],
        [0.3168, 0.6832],
        [0.3600, 0.6400],
        [0.3713, 0.6287],
        [0.3783, 0.6217],
        [0.4301, 0.5699],
        [0.4207, 0.5793],
        [0.3856, 0.6144],
        [0.2959, 0.7041],
        [0.4386, 0.5614],
        [0.3381, 0.6619],
        [0.3714, 0.6286],
        [0.2438, 0.7562]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4229, 0.5771],
        [0.4264, 0.5736],
        [0.4643, 0.5357],
        [0.3545, 0.6455],
        [0.4245, 0.5755],
        [0.4067, 0.5933],
        [0.4241, 0.5759],
        [0.4246, 0.5754],
        [0.3771, 0.6229],
        [0.3959, 0.6041],
        [0.4266, 0.5734],
        [0.4128, 0.5872],
        [0.3735, 0.6265],
        [0.3974, 0.6026]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 19 [   0/390]  Loss: 0.2513 (0.251)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.742e-02
Train: 19 [  50/390]  Loss: 0.2061 (0.348)  Acc@1: 95.3125 (87.6532)  Acc@5: 100.0000 (99.5404)LR: 1.742e-02
Train: 19 [ 100/390]  Loss: 0.3299 (0.349)  Acc@1: 85.9375 (87.7166)  Acc@5: 100.0000 (99.5978)LR: 1.742e-02
Train: 19 [ 150/390]  Loss: 0.2600 (0.356)  Acc@1: 90.6250 (87.5621)  Acc@5: 100.0000 (99.5550)LR: 1.742e-02
Train: 19 [ 200/390]  Loss: 0.2777 (0.347)  Acc@1: 92.1875 (87.9275)  Acc@5: 100.0000 (99.6191)LR: 1.742e-02
Train: 19 [ 250/390]  Loss: 0.2061 (0.348)  Acc@1: 92.1875 (87.8175)  Acc@5: 100.0000 (99.6016)LR: 1.742e-02
Train: 19 [ 300/390]  Loss: 0.3945 (0.353)  Acc@1: 85.9375 (87.6246)  Acc@5: 100.0000 (99.5951)LR: 1.742e-02
Train: 19 [ 350/390]  Loss: 0.3758 (0.358)  Acc@1: 92.1875 (87.5356)  Acc@5: 98.4375 (99.5771)LR: 1.742e-02
Train: 19 [ 390/390]  Loss: 0.3841 (0.361)  Acc@1: 87.5000 (87.4520)  Acc@5: 100.0000 (99.5680)LR: 1.742e-02
train_acc 87.452000
Valid: 19 [   0/390]  Loss: 0.4915 (0.492)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 19 [  50/390]  Loss: 0.5209 (0.571)  Acc@1: 82.8125 (81.1887)  Acc@5: 100.0000 (98.9890)
Valid: 19 [ 100/390]  Loss: 0.5267 (0.560)  Acc@1: 85.9375 (81.4047)  Acc@5: 98.4375 (99.0563)
Valid: 19 [ 150/390]  Loss: 0.5585 (0.560)  Acc@1: 81.2500 (81.3224)  Acc@5: 98.4375 (98.9652)
Valid: 19 [ 200/390]  Loss: 0.4031 (0.556)  Acc@1: 87.5000 (81.6154)  Acc@5: 100.0000 (99.0050)
Valid: 19 [ 250/390]  Loss: 0.5029 (0.560)  Acc@1: 81.2500 (81.6484)  Acc@5: 98.4375 (98.9355)
Valid: 19 [ 300/390]  Loss: 0.5112 (0.552)  Acc@1: 87.5000 (81.7016)  Acc@5: 100.0000 (99.0500)
Valid: 19 [ 350/390]  Loss: 0.3809 (0.548)  Acc@1: 85.9375 (81.9088)  Acc@5: 100.0000 (99.0696)
Valid: 19 [ 390/390]  Loss: 0.9596 (0.552)  Acc@1: 65.0000 (81.8320)  Acc@5: 97.5000 (99.0240)
valid_acc 81.832000
epoch = 19   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3415, 0.6585],
        [0.3722, 0.6278],
        [0.3080, 0.6920],
        [0.3535, 0.6465],
        [0.3661, 0.6339],
        [0.3728, 0.6272],
        [0.4263, 0.5737],
        [0.4209, 0.5791],
        [0.3812, 0.6188],
        [0.2910, 0.7090],
        [0.4392, 0.5608],
        [0.3395, 0.6605],
        [0.3724, 0.6276],
        [0.2332, 0.7668]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4223, 0.5777],
        [0.4224, 0.5776],
        [0.4615, 0.5385],
        [0.3507, 0.6493],
        [0.4212, 0.5788],
        [0.3960, 0.6040],
        [0.4232, 0.5768],
        [0.4144, 0.5856],
        [0.3685, 0.6315],
        [0.3886, 0.6114],
        [0.4235, 0.5765],
        [0.4159, 0.5841],
        [0.3694, 0.6306],
        [0.4016, 0.5984]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 20 [   0/390]  Loss: 0.2929 (0.293)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 1.671e-02
Train: 20 [  50/390]  Loss: 0.3671 (0.323)  Acc@1: 89.0625 (88.7868)  Acc@5: 100.0000 (99.6017)LR: 1.671e-02
Train: 20 [ 100/390]  Loss: 0.4714 (0.323)  Acc@1: 82.8125 (88.8304)  Acc@5: 100.0000 (99.7061)LR: 1.671e-02
Train: 20 [ 150/390]  Loss: 0.2558 (0.323)  Acc@1: 92.1875 (88.8452)  Acc@5: 100.0000 (99.7103)LR: 1.671e-02
Train: 20 [ 200/390]  Loss: 0.1314 (0.326)  Acc@1: 96.8750 (88.8604)  Acc@5: 100.0000 (99.6735)LR: 1.671e-02
Train: 20 [ 250/390]  Loss: 0.3443 (0.333)  Acc@1: 87.5000 (88.5147)  Acc@5: 98.4375 (99.6452)LR: 1.671e-02
Train: 20 [ 300/390]  Loss: 0.6383 (0.336)  Acc@1: 76.5625 (88.3409)  Acc@5: 98.4375 (99.6470)LR: 1.671e-02
Train: 20 [ 350/390]  Loss: 0.4393 (0.340)  Acc@1: 84.3750 (88.1855)  Acc@5: 98.4375 (99.6305)LR: 1.671e-02
Train: 20 [ 390/390]  Loss: 0.2810 (0.339)  Acc@1: 90.0000 (88.2000)  Acc@5: 100.0000 (99.6240)LR: 1.671e-02
train_acc 88.200000
Valid: 20 [   0/390]  Loss: 0.7279 (0.728)  Acc@1: 82.8125 (82.8125)  Acc@5: 93.7500 (93.7500)
Valid: 20 [  50/390]  Loss: 0.6244 (0.530)  Acc@1: 79.6875 (83.7316)  Acc@5: 96.8750 (98.8664)
Valid: 20 [ 100/390]  Loss: 0.3247 (0.508)  Acc@1: 90.6250 (84.0811)  Acc@5: 100.0000 (99.0254)
Valid: 20 [ 150/390]  Loss: 0.3243 (0.506)  Acc@1: 85.9375 (83.7127)  Acc@5: 100.0000 (99.1825)
Valid: 20 [ 200/390]  Loss: 0.5775 (0.503)  Acc@1: 76.5625 (83.6443)  Acc@5: 100.0000 (99.2071)
Valid: 20 [ 250/390]  Loss: 0.5083 (0.503)  Acc@1: 87.5000 (83.7089)  Acc@5: 98.4375 (99.2219)
Valid: 20 [ 300/390]  Loss: 0.3364 (0.500)  Acc@1: 90.6250 (83.7417)  Acc@5: 98.4375 (99.2733)
Valid: 20 [ 350/390]  Loss: 0.7985 (0.498)  Acc@1: 76.5625 (83.8408)  Acc@5: 96.8750 (99.2477)
Valid: 20 [ 390/390]  Loss: 0.3250 (0.494)  Acc@1: 95.0000 (83.9160)  Acc@5: 97.5000 (99.2560)
valid_acc 83.916000
epoch = 20   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3342, 0.6658],
        [0.3648, 0.6352],
        [0.3002, 0.6998],
        [0.3519, 0.6481],
        [0.3631, 0.6369],
        [0.3760, 0.6240],
        [0.4287, 0.5713],
        [0.4301, 0.5699],
        [0.3741, 0.6259],
        [0.2870, 0.7130],
        [0.4476, 0.5524],
        [0.3381, 0.6619],
        [0.3707, 0.6293],
        [0.2228, 0.7772]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4160, 0.5840],
        [0.4126, 0.5874],
        [0.4629, 0.5371],
        [0.3471, 0.6529],
        [0.4161, 0.5839],
        [0.3939, 0.6061],
        [0.4194, 0.5806],
        [0.4145, 0.5855],
        [0.3673, 0.6327],
        [0.3801, 0.6199],
        [0.4175, 0.5825],
        [0.4072, 0.5928],
        [0.3651, 0.6349],
        [0.3991, 0.6009]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 21 [   0/390]  Loss: 0.4807 (0.481)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.598e-02
Train: 21 [  50/390]  Loss: 0.1554 (0.316)  Acc@1: 95.3125 (89.0012)  Acc@5: 100.0000 (99.7549)LR: 1.598e-02
Train: 21 [ 100/390]  Loss: 0.3659 (0.325)  Acc@1: 90.6250 (88.7686)  Acc@5: 98.4375 (99.6287)LR: 1.598e-02
Train: 21 [ 150/390]  Loss: 0.3029 (0.320)  Acc@1: 87.5000 (88.8555)  Acc@5: 100.0000 (99.6482)LR: 1.598e-02
Train: 21 [ 200/390]  Loss: 0.3119 (0.322)  Acc@1: 89.0625 (88.9381)  Acc@5: 100.0000 (99.6191)LR: 1.598e-02
Train: 21 [ 250/390]  Loss: 0.2363 (0.321)  Acc@1: 92.1875 (88.9940)  Acc@5: 100.0000 (99.6389)LR: 1.598e-02
Train: 21 [ 300/390]  Loss: 0.2112 (0.323)  Acc@1: 92.1875 (88.8289)  Acc@5: 100.0000 (99.6574)LR: 1.598e-02
Train: 21 [ 350/390]  Loss: 0.4792 (0.325)  Acc@1: 85.9375 (88.7464)  Acc@5: 96.8750 (99.6572)LR: 1.598e-02
Train: 21 [ 390/390]  Loss: 0.3012 (0.329)  Acc@1: 85.0000 (88.6240)  Acc@5: 100.0000 (99.6440)LR: 1.598e-02
train_acc 88.624000
Valid: 21 [   0/390]  Loss: 0.5508 (0.551)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 21 [  50/390]  Loss: 0.3096 (0.467)  Acc@1: 87.5000 (84.6814)  Acc@5: 100.0000 (99.2034)
Valid: 21 [ 100/390]  Loss: 0.4813 (0.480)  Acc@1: 82.8125 (84.3595)  Acc@5: 100.0000 (99.1801)
Valid: 21 [ 150/390]  Loss: 0.3846 (0.479)  Acc@1: 85.9375 (84.2819)  Acc@5: 100.0000 (99.1618)
Valid: 21 [ 200/390]  Loss: 0.4852 (0.482)  Acc@1: 79.6875 (84.1651)  Acc@5: 100.0000 (99.1138)
Valid: 21 [ 250/390]  Loss: 0.5253 (0.485)  Acc@1: 82.8125 (84.0202)  Acc@5: 100.0000 (99.1160)
Valid: 21 [ 300/390]  Loss: 0.4803 (0.486)  Acc@1: 82.8125 (83.9701)  Acc@5: 100.0000 (99.1487)
Valid: 21 [ 350/390]  Loss: 0.5119 (0.487)  Acc@1: 84.3750 (83.9744)  Acc@5: 100.0000 (99.1809)
Valid: 21 [ 390/390]  Loss: 0.3825 (0.491)  Acc@1: 87.5000 (83.8680)  Acc@5: 100.0000 (99.1600)
valid_acc 83.868000
epoch = 21   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3266, 0.6734],
        [0.3599, 0.6401],
        [0.2912, 0.7088],
        [0.3456, 0.6544],
        [0.3642, 0.6358],
        [0.3767, 0.6233],
        [0.4248, 0.5752],
        [0.4340, 0.5660],
        [0.3682, 0.6318],
        [0.2854, 0.7146],
        [0.4503, 0.5497],
        [0.3342, 0.6658],
        [0.3730, 0.6270],
        [0.2138, 0.7862]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4151, 0.5849],
        [0.4121, 0.5879],
        [0.4633, 0.5367],
        [0.3405, 0.6595],
        [0.4118, 0.5882],
        [0.3917, 0.6083],
        [0.4138, 0.5862],
        [0.4117, 0.5883],
        [0.3643, 0.6357],
        [0.3734, 0.6266],
        [0.4135, 0.5865],
        [0.4038, 0.5962],
        [0.3598, 0.6402],
        [0.3996, 0.6004]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 22 [   0/390]  Loss: 0.3827 (0.383)  Acc@1: 84.3750 (84.3750)  Acc@5: 98.4375 (98.4375)LR: 1.525e-02
Train: 22 [  50/390]  Loss: 0.1446 (0.289)  Acc@1: 95.3125 (90.3493)  Acc@5: 100.0000 (99.6630)LR: 1.525e-02
Train: 22 [ 100/390]  Loss: 0.3394 (0.300)  Acc@1: 87.5000 (89.5885)  Acc@5: 100.0000 (99.7061)LR: 1.525e-02
Train: 22 [ 150/390]  Loss: 0.3165 (0.299)  Acc@1: 87.5000 (89.4557)  Acc@5: 100.0000 (99.7620)LR: 1.525e-02
Train: 22 [ 200/390]  Loss: 0.1556 (0.304)  Acc@1: 100.0000 (89.3890)  Acc@5: 100.0000 (99.7590)LR: 1.525e-02
Train: 22 [ 250/390]  Loss: 0.2769 (0.305)  Acc@1: 87.5000 (89.2119)  Acc@5: 100.0000 (99.7634)LR: 1.525e-02
Train: 22 [ 300/390]  Loss: 0.4775 (0.313)  Acc@1: 82.8125 (88.9950)  Acc@5: 100.0000 (99.7353)LR: 1.525e-02
Train: 22 [ 350/390]  Loss: 0.2276 (0.314)  Acc@1: 92.1875 (89.0714)  Acc@5: 100.0000 (99.7418)LR: 1.525e-02
Train: 22 [ 390/390]  Loss: 0.1793 (0.315)  Acc@1: 95.0000 (89.0520)  Acc@5: 100.0000 (99.7440)LR: 1.525e-02
train_acc 89.052000
Valid: 22 [   0/390]  Loss: 0.5629 (0.563)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 22 [  50/390]  Loss: 0.2910 (0.565)  Acc@1: 90.6250 (82.2610)  Acc@5: 100.0000 (99.3260)
Valid: 22 [ 100/390]  Loss: 0.7907 (0.565)  Acc@1: 84.3750 (82.6269)  Acc@5: 98.4375 (99.1646)
Valid: 22 [ 150/390]  Loss: 0.6346 (0.582)  Acc@1: 78.1250 (82.1192)  Acc@5: 100.0000 (99.0791)
Valid: 22 [ 200/390]  Loss: 0.5474 (0.571)  Acc@1: 81.2500 (82.3850)  Acc@5: 98.4375 (99.1294)
Valid: 22 [ 250/390]  Loss: 0.5573 (0.568)  Acc@1: 76.5625 (82.2958)  Acc@5: 96.8750 (99.0974)
Valid: 22 [ 300/390]  Loss: 0.2953 (0.569)  Acc@1: 87.5000 (82.2519)  Acc@5: 100.0000 (99.0708)
Valid: 22 [ 350/390]  Loss: 0.6275 (0.570)  Acc@1: 78.1250 (82.2382)  Acc@5: 100.0000 (99.0607)
Valid: 22 [ 390/390]  Loss: 1.171 (0.572)  Acc@1: 70.0000 (82.2760)  Acc@5: 92.5000 (99.0400)
valid_acc 82.276000
epoch = 22   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3225, 0.6775],
        [0.3491, 0.6509],
        [0.2848, 0.7152],
        [0.3392, 0.6608],
        [0.3617, 0.6383],
        [0.3747, 0.6253],
        [0.4203, 0.5797],
        [0.4371, 0.5629],
        [0.3647, 0.6353],
        [0.2811, 0.7189],
        [0.4516, 0.5484],
        [0.3317, 0.6683],
        [0.3733, 0.6267],
        [0.2066, 0.7934]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4171, 0.5829],
        [0.4098, 0.5902],
        [0.4604, 0.5396],
        [0.3380, 0.6620],
        [0.4123, 0.5877],
        [0.3872, 0.6128],
        [0.4114, 0.5886],
        [0.4078, 0.5922],
        [0.3580, 0.6420],
        [0.3709, 0.6291],
        [0.4108, 0.5892],
        [0.4013, 0.5987],
        [0.3555, 0.6445],
        [0.4043, 0.5957]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 23 [   0/390]  Loss: 0.2249 (0.225)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.450e-02
Train: 23 [  50/390]  Loss: 0.1724 (0.296)  Acc@1: 95.3125 (89.3382)  Acc@5: 100.0000 (99.7855)LR: 1.450e-02
Train: 23 [ 100/390]  Loss: 0.2206 (0.303)  Acc@1: 92.1875 (89.1089)  Acc@5: 100.0000 (99.7525)LR: 1.450e-02
Train: 23 [ 150/390]  Loss: 0.2986 (0.295)  Acc@1: 85.9375 (89.3936)  Acc@5: 100.0000 (99.7206)LR: 1.450e-02
Train: 23 [ 200/390]  Loss: 0.2074 (0.297)  Acc@1: 90.6250 (89.2879)  Acc@5: 100.0000 (99.7435)LR: 1.450e-02
Train: 23 [ 250/390]  Loss: 0.3696 (0.297)  Acc@1: 89.0625 (89.3675)  Acc@5: 98.4375 (99.7323)LR: 1.450e-02
Train: 23 [ 300/390]  Loss: 0.3240 (0.295)  Acc@1: 90.6250 (89.3792)  Acc@5: 100.0000 (99.7456)LR: 1.450e-02
Train: 23 [ 350/390]  Loss: 0.2713 (0.300)  Acc@1: 90.6250 (89.2317)  Acc@5: 100.0000 (99.7463)LR: 1.450e-02
Train: 23 [ 390/390]  Loss: 0.2270 (0.301)  Acc@1: 90.0000 (89.1880)  Acc@5: 100.0000 (99.7480)LR: 1.450e-02
train_acc 89.188000
Valid: 23 [   0/390]  Loss: 0.5033 (0.503)  Acc@1: 79.6875 (79.6875)  Acc@5: 98.4375 (98.4375)
Valid: 23 [  50/390]  Loss: 0.5426 (0.475)  Acc@1: 87.5000 (84.9877)  Acc@5: 98.4375 (99.0502)
Valid: 23 [ 100/390]  Loss: 0.6391 (0.474)  Acc@1: 82.8125 (84.6999)  Acc@5: 100.0000 (99.2574)
Valid: 23 [ 150/390]  Loss: 0.2219 (0.475)  Acc@1: 90.6250 (84.4785)  Acc@5: 100.0000 (99.2446)
Valid: 23 [ 200/390]  Loss: 0.3225 (0.472)  Acc@1: 92.1875 (84.4683)  Acc@5: 100.0000 (99.3004)
Valid: 23 [ 250/390]  Loss: 0.4992 (0.473)  Acc@1: 81.2500 (84.4808)  Acc@5: 100.0000 (99.3028)
Valid: 23 [ 300/390]  Loss: 0.5321 (0.473)  Acc@1: 82.8125 (84.5515)  Acc@5: 96.8750 (99.2940)
Valid: 23 [ 350/390]  Loss: 0.6373 (0.472)  Acc@1: 78.1250 (84.6154)  Acc@5: 98.4375 (99.2922)
Valid: 23 [ 390/390]  Loss: 0.7536 (0.473)  Acc@1: 80.0000 (84.5520)  Acc@5: 97.5000 (99.2720)
valid_acc 84.552000
epoch = 23   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3145, 0.6855],
        [0.3419, 0.6581],
        [0.2739, 0.7261],
        [0.3323, 0.6677],
        [0.3569, 0.6431],
        [0.3701, 0.6299],
        [0.4154, 0.5846],
        [0.4426, 0.5574],
        [0.3657, 0.6343],
        [0.2794, 0.7206],
        [0.4539, 0.5461],
        [0.3301, 0.6699],
        [0.3746, 0.6254],
        [0.1974, 0.8026]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4168, 0.5832],
        [0.4057, 0.5943],
        [0.4608, 0.5392],
        [0.3313, 0.6687],
        [0.4097, 0.5903],
        [0.3823, 0.6177],
        [0.4056, 0.5944],
        [0.4060, 0.5940],
        [0.3518, 0.6482],
        [0.3621, 0.6379],
        [0.4094, 0.5906],
        [0.3976, 0.6024],
        [0.3488, 0.6512],
        [0.4047, 0.5953]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 24 [   0/390]  Loss: 0.2198 (0.220)  Acc@1: 93.7500 (93.7500)  Acc@5: 98.4375 (98.4375)LR: 1.375e-02
Train: 24 [  50/390]  Loss: 0.2990 (0.289)  Acc@1: 90.6250 (89.6446)  Acc@5: 100.0000 (99.6630)LR: 1.375e-02
Train: 24 [ 100/390]  Loss: 0.1940 (0.289)  Acc@1: 93.7500 (90.0526)  Acc@5: 100.0000 (99.6597)LR: 1.375e-02
Train: 24 [ 150/390]  Loss: 0.4480 (0.291)  Acc@1: 87.5000 (89.8696)  Acc@5: 98.4375 (99.6689)LR: 1.375e-02
Train: 24 [ 200/390]  Loss: 0.1510 (0.282)  Acc@1: 93.7500 (90.2208)  Acc@5: 100.0000 (99.6657)LR: 1.375e-02
Train: 24 [ 250/390]  Loss: 0.1448 (0.290)  Acc@1: 92.1875 (89.9340)  Acc@5: 100.0000 (99.6638)LR: 1.375e-02
Train: 24 [ 300/390]  Loss: 0.2936 (0.289)  Acc@1: 90.6250 (90.0332)  Acc@5: 100.0000 (99.6574)LR: 1.375e-02
Train: 24 [ 350/390]  Loss: 0.2399 (0.289)  Acc@1: 93.7500 (90.0953)  Acc@5: 100.0000 (99.6350)LR: 1.375e-02
Train: 24 [ 390/390]  Loss: 0.3879 (0.291)  Acc@1: 80.0000 (90.0520)  Acc@5: 100.0000 (99.6320)LR: 1.375e-02
train_acc 90.052000
Valid: 24 [   0/390]  Loss: 0.2550 (0.255)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)
Valid: 24 [  50/390]  Loss: 0.2765 (0.441)  Acc@1: 89.0625 (86.0294)  Acc@5: 100.0000 (99.4179)
Valid: 24 [ 100/390]  Loss: 0.5429 (0.458)  Acc@1: 82.8125 (85.2723)  Acc@5: 100.0000 (99.3812)
Valid: 24 [ 150/390]  Loss: 0.4162 (0.465)  Acc@1: 84.3750 (85.0166)  Acc@5: 98.4375 (99.3274)
Valid: 24 [ 200/390]  Loss: 0.7520 (0.470)  Acc@1: 82.8125 (84.8103)  Acc@5: 100.0000 (99.2771)
Valid: 24 [ 250/390]  Loss: 0.3807 (0.465)  Acc@1: 87.5000 (84.9913)  Acc@5: 100.0000 (99.3152)
Valid: 24 [ 300/390]  Loss: 0.3678 (0.465)  Acc@1: 85.9375 (84.9356)  Acc@5: 98.4375 (99.3148)
Valid: 24 [ 350/390]  Loss: 0.4399 (0.464)  Acc@1: 89.0625 (84.9537)  Acc@5: 98.4375 (99.2967)
Valid: 24 [ 390/390]  Loss: 0.5349 (0.463)  Acc@1: 82.5000 (84.9400)  Acc@5: 100.0000 (99.2760)
valid_acc 84.940000
epoch = 24   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3082, 0.6918],
        [0.3338, 0.6662],
        [0.2669, 0.7331],
        [0.3286, 0.6714],
        [0.3573, 0.6427],
        [0.3740, 0.6260],
        [0.4133, 0.5867],
        [0.4483, 0.5517],
        [0.3619, 0.6381],
        [0.2724, 0.7276],
        [0.4551, 0.5449],
        [0.3297, 0.6703],
        [0.3723, 0.6277],
        [0.1918, 0.8082]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4154, 0.5846],
        [0.4006, 0.5994],
        [0.4585, 0.5415],
        [0.3267, 0.6733],
        [0.4111, 0.5889],
        [0.3738, 0.6262],
        [0.3986, 0.6014],
        [0.4022, 0.5978],
        [0.3442, 0.6558],
        [0.3566, 0.6434],
        [0.4067, 0.5933],
        [0.3946, 0.6054],
        [0.3417, 0.6583],
        [0.4050, 0.5950]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 25 [   0/390]  Loss: 0.1904 (0.190)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.300e-02
Train: 25 [  50/390]  Loss: 0.3410 (0.259)  Acc@1: 90.6250 (90.8701)  Acc@5: 100.0000 (99.8775)LR: 1.300e-02
Train: 25 [ 100/390]  Loss: 0.2406 (0.261)  Acc@1: 90.6250 (90.8261)  Acc@5: 100.0000 (99.8917)LR: 1.300e-02
Train: 25 [ 150/390]  Loss: 0.3188 (0.269)  Acc@1: 87.5000 (90.6974)  Acc@5: 100.0000 (99.8655)LR: 1.300e-02
Train: 25 [ 200/390]  Loss: 0.3598 (0.272)  Acc@1: 87.5000 (90.5006)  Acc@5: 100.0000 (99.8523)LR: 1.300e-02
Train: 25 [ 250/390]  Loss: 0.4837 (0.276)  Acc@1: 85.9375 (90.3200)  Acc@5: 98.4375 (99.8319)LR: 1.300e-02
Train: 25 [ 300/390]  Loss: 0.1627 (0.278)  Acc@1: 93.7500 (90.2461)  Acc@5: 100.0000 (99.8027)LR: 1.300e-02
Train: 25 [ 350/390]  Loss: 0.3015 (0.278)  Acc@1: 90.6250 (90.2600)  Acc@5: 98.4375 (99.7819)LR: 1.300e-02
Train: 25 [ 390/390]  Loss: 0.2431 (0.277)  Acc@1: 90.0000 (90.2840)  Acc@5: 100.0000 (99.7840)LR: 1.300e-02
train_acc 90.284000
Valid: 25 [   0/390]  Loss: 0.4775 (0.478)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 25 [  50/390]  Loss: 0.3545 (0.472)  Acc@1: 87.5000 (84.5895)  Acc@5: 100.0000 (99.4179)
Valid: 25 [ 100/390]  Loss: 0.5340 (0.475)  Acc@1: 84.3750 (84.6071)  Acc@5: 100.0000 (99.2420)
Valid: 25 [ 150/390]  Loss: 0.6002 (0.476)  Acc@1: 81.2500 (84.5716)  Acc@5: 100.0000 (99.2757)
Valid: 25 [ 200/390]  Loss: 0.4471 (0.475)  Acc@1: 87.5000 (84.6549)  Acc@5: 100.0000 (99.2537)
Valid: 25 [ 250/390]  Loss: 0.3020 (0.475)  Acc@1: 92.1875 (84.6489)  Acc@5: 100.0000 (99.2717)
Valid: 25 [ 300/390]  Loss: 0.6354 (0.478)  Acc@1: 76.5625 (84.5723)  Acc@5: 98.4375 (99.2681)
Valid: 25 [ 350/390]  Loss: 0.3319 (0.471)  Acc@1: 87.5000 (84.7623)  Acc@5: 100.0000 (99.3278)
Valid: 25 [ 390/390]  Loss: 0.6981 (0.468)  Acc@1: 82.5000 (84.8720)  Acc@5: 92.5000 (99.3360)
valid_acc 84.872000
epoch = 25   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3069, 0.6931],
        [0.3247, 0.6753],
        [0.2584, 0.7416],
        [0.3235, 0.6765],
        [0.3563, 0.6437],
        [0.3725, 0.6275],
        [0.4134, 0.5866],
        [0.4544, 0.5456],
        [0.3606, 0.6394],
        [0.2699, 0.7301],
        [0.4613, 0.5387],
        [0.3323, 0.6677],
        [0.3787, 0.6213],
        [0.1867, 0.8133]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4149, 0.5851],
        [0.3994, 0.6006],
        [0.4601, 0.5399],
        [0.3264, 0.6736],
        [0.4076, 0.5924],
        [0.3715, 0.6285],
        [0.3921, 0.6079],
        [0.3967, 0.6033],
        [0.3407, 0.6593],
        [0.3528, 0.6472],
        [0.4068, 0.5932],
        [0.3899, 0.6101],
        [0.3356, 0.6644],
        [0.4070, 0.5930]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 26 [   0/390]  Loss: 0.3568 (0.357)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 1.225e-02
Train: 26 [  50/390]  Loss: 0.2749 (0.258)  Acc@1: 89.0625 (90.8701)  Acc@5: 100.0000 (99.8775)LR: 1.225e-02
Train: 26 [ 100/390]  Loss: 0.2722 (0.249)  Acc@1: 90.6250 (91.1819)  Acc@5: 100.0000 (99.8453)LR: 1.225e-02
Train: 26 [ 150/390]  Loss: 0.2568 (0.251)  Acc@1: 87.5000 (91.2045)  Acc@5: 100.0000 (99.8655)LR: 1.225e-02
Train: 26 [ 200/390]  Loss: 0.3600 (0.254)  Acc@1: 90.6250 (91.0603)  Acc@5: 100.0000 (99.8912)LR: 1.225e-02
Train: 26 [ 250/390]  Loss: 0.3496 (0.256)  Acc@1: 85.9375 (90.8678)  Acc@5: 100.0000 (99.8817)LR: 1.225e-02
Train: 26 [ 300/390]  Loss: 0.1496 (0.260)  Acc@1: 96.8750 (90.7340)  Acc@5: 100.0000 (99.8598)LR: 1.225e-02
Train: 26 [ 350/390]  Loss: 0.2153 (0.258)  Acc@1: 93.7500 (90.7229)  Acc@5: 100.0000 (99.8665)LR: 1.225e-02
Train: 26 [ 390/390]  Loss: 0.2794 (0.259)  Acc@1: 90.0000 (90.7480)  Acc@5: 100.0000 (99.8720)LR: 1.225e-02
train_acc 90.748000
Valid: 26 [   0/390]  Loss: 0.4236 (0.424)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 26 [  50/390]  Loss: 0.3640 (0.490)  Acc@1: 87.5000 (84.6507)  Acc@5: 100.0000 (99.3873)
Valid: 26 [ 100/390]  Loss: 0.3088 (0.473)  Acc@1: 84.3750 (84.9010)  Acc@5: 100.0000 (99.2574)
Valid: 26 [ 150/390]  Loss: 0.2360 (0.471)  Acc@1: 93.7500 (84.9959)  Acc@5: 100.0000 (99.2757)
Valid: 26 [ 200/390]  Loss: 0.7467 (0.483)  Acc@1: 78.1250 (84.7948)  Acc@5: 100.0000 (99.2460)
Valid: 26 [ 250/390]  Loss: 0.4918 (0.485)  Acc@1: 82.8125 (84.7174)  Acc@5: 100.0000 (99.2592)
Valid: 26 [ 300/390]  Loss: 0.4359 (0.487)  Acc@1: 82.8125 (84.7436)  Acc@5: 98.4375 (99.2265)
Valid: 26 [ 350/390]  Loss: 0.5963 (0.485)  Acc@1: 84.3750 (84.7979)  Acc@5: 96.8750 (99.2121)
Valid: 26 [ 390/390]  Loss: 0.3355 (0.485)  Acc@1: 92.5000 (84.8080)  Acc@5: 100.0000 (99.1920)
valid_acc 84.808000
epoch = 26   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3036, 0.6964],
        [0.3174, 0.6826],
        [0.2510, 0.7490],
        [0.3167, 0.6833],
        [0.3519, 0.6481],
        [0.3762, 0.6238],
        [0.4097, 0.5903],
        [0.4599, 0.5401],
        [0.3628, 0.6372],
        [0.2681, 0.7319],
        [0.4643, 0.5357],
        [0.3288, 0.6712],
        [0.3805, 0.6195],
        [0.1813, 0.8187]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4119, 0.5881],
        [0.3947, 0.6053],
        [0.4523, 0.5477],
        [0.3235, 0.6765],
        [0.4070, 0.5930],
        [0.3687, 0.6313],
        [0.3875, 0.6125],
        [0.3883, 0.6117],
        [0.3358, 0.6642],
        [0.3517, 0.6483],
        [0.4060, 0.5940],
        [0.3820, 0.6180],
        [0.3283, 0.6717],
        [0.4072, 0.5928]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 27 [   0/390]  Loss: 0.1147 (0.115)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 1.150e-02
Train: 27 [  50/390]  Loss: 0.2125 (0.232)  Acc@1: 95.3125 (91.6667)  Acc@5: 100.0000 (99.9081)LR: 1.150e-02
Train: 27 [ 100/390]  Loss: 0.2040 (0.230)  Acc@1: 93.7500 (91.8626)  Acc@5: 100.0000 (99.8608)LR: 1.150e-02
Train: 27 [ 150/390]  Loss: 0.1761 (0.237)  Acc@1: 93.7500 (91.4942)  Acc@5: 100.0000 (99.8448)LR: 1.150e-02
Train: 27 [ 200/390]  Loss: 0.2400 (0.239)  Acc@1: 93.7500 (91.3557)  Acc@5: 100.0000 (99.8445)LR: 1.150e-02
Train: 27 [ 250/390]  Loss: 0.5077 (0.244)  Acc@1: 85.9375 (91.2600)  Acc@5: 100.0000 (99.8319)LR: 1.150e-02
Train: 27 [ 300/390]  Loss: 0.3221 (0.253)  Acc@1: 90.6250 (91.0403)  Acc@5: 100.0000 (99.8183)LR: 1.150e-02
Train: 27 [ 350/390]  Loss: 0.1492 (0.252)  Acc@1: 95.3125 (91.1414)  Acc@5: 100.0000 (99.8264)LR: 1.150e-02
Train: 27 [ 390/390]  Loss: 0.2796 (0.250)  Acc@1: 87.5000 (91.2120)  Acc@5: 100.0000 (99.8360)LR: 1.150e-02
train_acc 91.212000
Valid: 27 [   0/390]  Loss: 0.2304 (0.230)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 27 [  50/390]  Loss: 0.6118 (0.434)  Acc@1: 79.6875 (85.7843)  Acc@5: 100.0000 (99.3566)
Valid: 27 [ 100/390]  Loss: 0.6282 (0.443)  Acc@1: 79.6875 (85.9839)  Acc@5: 98.4375 (99.4585)
Valid: 27 [ 150/390]  Loss: 0.2662 (0.459)  Acc@1: 89.0625 (85.5236)  Acc@5: 100.0000 (99.3377)
Valid: 27 [ 200/390]  Loss: 0.5370 (0.459)  Acc@1: 79.6875 (85.5100)  Acc@5: 100.0000 (99.3626)
Valid: 27 [ 250/390]  Loss: 0.7034 (0.458)  Acc@1: 79.6875 (85.5142)  Acc@5: 100.0000 (99.3588)
Valid: 27 [ 300/390]  Loss: 0.4720 (0.450)  Acc@1: 82.8125 (85.6935)  Acc@5: 100.0000 (99.3875)
Valid: 27 [ 350/390]  Loss: 0.3698 (0.450)  Acc@1: 90.6250 (85.6838)  Acc@5: 100.0000 (99.4124)
Valid: 27 [ 390/390]  Loss: 0.4979 (0.449)  Acc@1: 85.0000 (85.7040)  Acc@5: 100.0000 (99.4120)
valid_acc 85.704000
epoch = 27   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2968, 0.7032],
        [0.3097, 0.6903],
        [0.2391, 0.7609],
        [0.3091, 0.6909],
        [0.3475, 0.6525],
        [0.3720, 0.6280],
        [0.4043, 0.5957],
        [0.4668, 0.5332],
        [0.3594, 0.6406],
        [0.2631, 0.7369],
        [0.4632, 0.5368],
        [0.3267, 0.6733],
        [0.3826, 0.6174],
        [0.1730, 0.8270]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4096, 0.5904],
        [0.3910, 0.6090],
        [0.4495, 0.5505],
        [0.3155, 0.6845],
        [0.4072, 0.5928],
        [0.3631, 0.6369],
        [0.3789, 0.6211],
        [0.3793, 0.6207],
        [0.3294, 0.6706],
        [0.3478, 0.6522],
        [0.3951, 0.6049],
        [0.3739, 0.6261],
        [0.3233, 0.6767],
        [0.4062, 0.5938]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 28 [   0/390]  Loss: 0.2772 (0.277)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 1.075e-02
Train: 28 [  50/390]  Loss: 0.09326 (0.248)  Acc@1: 96.8750 (90.8088)  Acc@5: 100.0000 (99.8162)LR: 1.075e-02
Train: 28 [ 100/390]  Loss: 0.3617 (0.239)  Acc@1: 87.5000 (91.5068)  Acc@5: 98.4375 (99.8144)LR: 1.075e-02
Train: 28 [ 150/390]  Loss: 0.1952 (0.233)  Acc@1: 95.3125 (91.7219)  Acc@5: 100.0000 (99.8344)LR: 1.075e-02
Train: 28 [ 200/390]  Loss: 0.1991 (0.231)  Acc@1: 93.7500 (91.6978)  Acc@5: 100.0000 (99.8368)LR: 1.075e-02
Train: 28 [ 250/390]  Loss: 0.3115 (0.234)  Acc@1: 87.5000 (91.5650)  Acc@5: 100.0000 (99.8568)LR: 1.075e-02
Train: 28 [ 300/390]  Loss: 0.1785 (0.239)  Acc@1: 92.1875 (91.3673)  Acc@5: 100.0000 (99.8650)LR: 1.075e-02
Train: 28 [ 350/390]  Loss: 0.5048 (0.241)  Acc@1: 79.6875 (91.3818)  Acc@5: 100.0000 (99.8709)LR: 1.075e-02
Train: 28 [ 390/390]  Loss: 0.3925 (0.243)  Acc@1: 82.5000 (91.3360)  Acc@5: 100.0000 (99.8640)LR: 1.075e-02
train_acc 91.336000
Valid: 28 [   0/390]  Loss: 0.4202 (0.420)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 28 [  50/390]  Loss: 0.6050 (0.434)  Acc@1: 79.6875 (86.4277)  Acc@5: 100.0000 (99.3566)
Valid: 28 [ 100/390]  Loss: 0.7466 (0.438)  Acc@1: 79.6875 (86.3552)  Acc@5: 93.7500 (99.3812)
Valid: 28 [ 150/390]  Loss: 0.5179 (0.431)  Acc@1: 79.6875 (86.2893)  Acc@5: 100.0000 (99.4516)
Valid: 28 [ 200/390]  Loss: 0.6592 (0.439)  Acc@1: 84.3750 (86.1940)  Acc@5: 95.3125 (99.3937)
Valid: 28 [ 250/390]  Loss: 0.5272 (0.438)  Acc@1: 81.2500 (86.2176)  Acc@5: 98.4375 (99.3775)
Valid: 28 [ 300/390]  Loss: 0.2218 (0.438)  Acc@1: 93.7500 (86.1555)  Acc@5: 100.0000 (99.3511)
Valid: 28 [ 350/390]  Loss: 0.4246 (0.438)  Acc@1: 81.2500 (86.1200)  Acc@5: 98.4375 (99.3412)
Valid: 28 [ 390/390]  Loss: 0.3205 (0.439)  Acc@1: 90.0000 (86.0760)  Acc@5: 100.0000 (99.3360)
valid_acc 86.076000
epoch = 28   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2899, 0.7101],
        [0.3032, 0.6968],
        [0.2313, 0.7687],
        [0.3032, 0.6968],
        [0.3465, 0.6535],
        [0.3741, 0.6259],
        [0.4067, 0.5933],
        [0.4733, 0.5267],
        [0.3572, 0.6428],
        [0.2605, 0.7395],
        [0.4643, 0.5357],
        [0.3257, 0.6743],
        [0.3818, 0.6182],
        [0.1675, 0.8325]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4130, 0.5870],
        [0.3849, 0.6151],
        [0.4476, 0.5524],
        [0.3121, 0.6879],
        [0.4049, 0.5951],
        [0.3588, 0.6412],
        [0.3732, 0.6268],
        [0.3759, 0.6241],
        [0.3273, 0.6727],
        [0.3416, 0.6584],
        [0.3936, 0.6064],
        [0.3692, 0.6308],
        [0.3181, 0.6819],
        [0.4061, 0.5939]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 29 [   0/390]  Loss: 0.2910 (0.291)  Acc@1: 92.1875 (92.1875)  Acc@5: 98.4375 (98.4375)LR: 1.002e-02
Train: 29 [  50/390]  Loss: 0.2566 (0.220)  Acc@1: 93.7500 (92.5551)  Acc@5: 100.0000 (99.7855)LR: 1.002e-02
Train: 29 [ 100/390]  Loss: 0.1687 (0.206)  Acc@1: 95.3125 (92.9610)  Acc@5: 100.0000 (99.8608)LR: 1.002e-02
Train: 29 [ 150/390]  Loss: 0.1017 (0.205)  Acc@1: 98.4375 (92.8704)  Acc@5: 100.0000 (99.8862)LR: 1.002e-02
Train: 29 [ 200/390]  Loss: 0.3690 (0.204)  Acc@1: 87.5000 (92.8483)  Acc@5: 100.0000 (99.9067)LR: 1.002e-02
Train: 29 [ 250/390]  Loss: 0.2744 (0.209)  Acc@1: 92.1875 (92.6917)  Acc@5: 100.0000 (99.9004)LR: 1.002e-02
Train: 29 [ 300/390]  Loss: 0.06853 (0.212)  Acc@1: 100.0000 (92.5768)  Acc@5: 100.0000 (99.9014)LR: 1.002e-02
Train: 29 [ 350/390]  Loss: 0.1217 (0.212)  Acc@1: 93.7500 (92.5970)  Acc@5: 100.0000 (99.8932)LR: 1.002e-02
Train: 29 [ 390/390]  Loss: 0.06817 (0.214)  Acc@1: 97.5000 (92.5120)  Acc@5: 100.0000 (99.8880)LR: 1.002e-02
train_acc 92.512000
Valid: 29 [   0/390]  Loss: 0.4937 (0.494)  Acc@1: 81.2500 (81.2500)  Acc@5: 100.0000 (100.0000)
Valid: 29 [  50/390]  Loss: 0.3916 (0.458)  Acc@1: 90.6250 (86.1826)  Acc@5: 98.4375 (99.2647)
Valid: 29 [ 100/390]  Loss: 0.2897 (0.450)  Acc@1: 85.9375 (86.3552)  Acc@5: 100.0000 (99.3193)
Valid: 29 [ 150/390]  Loss: 0.2689 (0.447)  Acc@1: 90.6250 (86.3825)  Acc@5: 100.0000 (99.3584)
Valid: 29 [ 200/390]  Loss: 0.1729 (0.436)  Acc@1: 90.6250 (86.5050)  Acc@5: 100.0000 (99.3937)
Valid: 29 [ 250/390]  Loss: 0.7581 (0.441)  Acc@1: 79.6875 (86.4791)  Acc@5: 100.0000 (99.3775)
Valid: 29 [ 300/390]  Loss: 0.3125 (0.440)  Acc@1: 89.0625 (86.4981)  Acc@5: 98.4375 (99.3459)
Valid: 29 [ 350/390]  Loss: 0.4624 (0.441)  Acc@1: 87.5000 (86.4316)  Acc@5: 98.4375 (99.3456)
Valid: 29 [ 390/390]  Loss: 0.5716 (0.441)  Acc@1: 85.0000 (86.4400)  Acc@5: 97.5000 (99.3200)
valid_acc 86.440000
epoch = 29   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2853, 0.7147],
        [0.2939, 0.7061],
        [0.2231, 0.7769],
        [0.2989, 0.7011],
        [0.3464, 0.6536],
        [0.3704, 0.6296],
        [0.4074, 0.5926],
        [0.4777, 0.5223],
        [0.3560, 0.6440],
        [0.2597, 0.7403],
        [0.4709, 0.5291],
        [0.3291, 0.6709],
        [0.3870, 0.6130],
        [0.1634, 0.8366]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4061, 0.5939],
        [0.3768, 0.6232],
        [0.4424, 0.5576],
        [0.3081, 0.6919],
        [0.3990, 0.6010],
        [0.3565, 0.6435],
        [0.3686, 0.6314],
        [0.3709, 0.6291],
        [0.3239, 0.6761],
        [0.3341, 0.6659],
        [0.3902, 0.6098],
        [0.3632, 0.6368],
        [0.3107, 0.6893],
        [0.3996, 0.6004]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 30 [   0/390]  Loss: 0.1015 (0.101)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 9.292e-03
Train: 30 [  50/390]  Loss: 0.09656 (0.185)  Acc@1: 96.8750 (93.3211)  Acc@5: 100.0000 (99.9694)LR: 9.292e-03
Train: 30 [ 100/390]  Loss: 0.1051 (0.196)  Acc@1: 95.3125 (93.0693)  Acc@5: 100.0000 (99.9381)LR: 9.292e-03
Train: 30 [ 150/390]  Loss: 0.09066 (0.194)  Acc@1: 98.4375 (93.2326)  Acc@5: 100.0000 (99.9276)LR: 9.292e-03
Train: 30 [ 200/390]  Loss: 0.1430 (0.197)  Acc@1: 96.8750 (93.0581)  Acc@5: 100.0000 (99.9223)LR: 9.292e-03
Train: 30 [ 250/390]  Loss: 0.3230 (0.201)  Acc@1: 90.6250 (92.8474)  Acc@5: 100.0000 (99.9377)LR: 9.292e-03
Train: 30 [ 300/390]  Loss: 0.1001 (0.207)  Acc@1: 96.8750 (92.6080)  Acc@5: 100.0000 (99.9169)LR: 9.292e-03
Train: 30 [ 350/390]  Loss: 0.2189 (0.210)  Acc@1: 92.1875 (92.4590)  Acc@5: 100.0000 (99.9110)LR: 9.292e-03
Train: 30 [ 390/390]  Loss: 0.3095 (0.214)  Acc@1: 87.5000 (92.3440)  Acc@5: 100.0000 (99.9000)LR: 9.292e-03
train_acc 92.344000
Valid: 30 [   0/390]  Loss: 0.6262 (0.626)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 30 [  50/390]  Loss: 0.3604 (0.428)  Acc@1: 89.0625 (86.6115)  Acc@5: 100.0000 (99.5098)
Valid: 30 [ 100/390]  Loss: 0.1925 (0.456)  Acc@1: 93.7500 (86.0613)  Acc@5: 100.0000 (99.4121)
Valid: 30 [ 150/390]  Loss: 0.3483 (0.463)  Acc@1: 90.6250 (86.0306)  Acc@5: 98.4375 (99.2964)
Valid: 30 [ 200/390]  Loss: 0.5200 (0.464)  Acc@1: 84.3750 (86.0152)  Acc@5: 100.0000 (99.3081)
Valid: 30 [ 250/390]  Loss: 0.6554 (0.465)  Acc@1: 79.6875 (85.9001)  Acc@5: 100.0000 (99.3028)
Valid: 30 [ 300/390]  Loss: 0.6424 (0.462)  Acc@1: 79.6875 (85.9531)  Acc@5: 100.0000 (99.2992)
Valid: 30 [ 350/390]  Loss: 0.4544 (0.461)  Acc@1: 85.9375 (85.9820)  Acc@5: 100.0000 (99.2922)
Valid: 30 [ 390/390]  Loss: 0.4564 (0.461)  Acc@1: 82.5000 (85.9960)  Acc@5: 100.0000 (99.2920)
valid_acc 85.996000
epoch = 30   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2810, 0.7190],
        [0.2854, 0.7146],
        [0.2175, 0.7825],
        [0.2934, 0.7066],
        [0.3464, 0.6536],
        [0.3703, 0.6297],
        [0.4034, 0.5966],
        [0.4845, 0.5155],
        [0.3588, 0.6412],
        [0.2572, 0.7428],
        [0.4730, 0.5270],
        [0.3335, 0.6665],
        [0.3912, 0.6088],
        [0.1602, 0.8398]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4064, 0.5936],
        [0.3712, 0.6288],
        [0.4320, 0.5680],
        [0.2988, 0.7012],
        [0.3971, 0.6029],
        [0.3474, 0.6526],
        [0.3653, 0.6347],
        [0.3655, 0.6345],
        [0.3167, 0.6833],
        [0.3305, 0.6695],
        [0.3851, 0.6149],
        [0.3560, 0.6440],
        [0.3042, 0.6958],
        [0.3987, 0.6013]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 31 [   0/390]  Loss: 0.3508 (0.351)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 8.583e-03
Train: 31 [  50/390]  Loss: 0.2042 (0.178)  Acc@1: 92.1875 (93.4130)  Acc@5: 100.0000 (99.9081)LR: 8.583e-03
Train: 31 [ 100/390]  Loss: 0.1125 (0.182)  Acc@1: 96.8750 (93.4561)  Acc@5: 100.0000 (99.9072)LR: 8.583e-03
Train: 31 [ 150/390]  Loss: 0.2242 (0.180)  Acc@1: 87.5000 (93.5948)  Acc@5: 100.0000 (99.8965)LR: 8.583e-03
Train: 31 [ 200/390]  Loss: 0.2268 (0.185)  Acc@1: 89.0625 (93.5323)  Acc@5: 100.0000 (99.9145)LR: 8.583e-03
Train: 31 [ 250/390]  Loss: 0.1938 (0.184)  Acc@1: 92.1875 (93.5446)  Acc@5: 100.0000 (99.9191)LR: 8.583e-03
Train: 31 [ 300/390]  Loss: 0.1066 (0.187)  Acc@1: 95.3125 (93.4749)  Acc@5: 100.0000 (99.9169)LR: 8.583e-03
Train: 31 [ 350/390]  Loss: 0.4899 (0.189)  Acc@1: 84.3750 (93.3627)  Acc@5: 98.4375 (99.9065)LR: 8.583e-03
Train: 31 [ 390/390]  Loss: 0.1701 (0.188)  Acc@1: 95.0000 (93.4200)  Acc@5: 100.0000 (99.9040)LR: 8.583e-03
train_acc 93.420000
Valid: 31 [   0/390]  Loss: 0.5702 (0.570)  Acc@1: 84.3750 (84.3750)  Acc@5: 98.4375 (98.4375)
Valid: 31 [  50/390]  Loss: 0.3526 (0.480)  Acc@1: 89.0625 (85.4167)  Acc@5: 100.0000 (99.3566)
Valid: 31 [ 100/390]  Loss: 0.5099 (0.455)  Acc@1: 81.2500 (86.2160)  Acc@5: 100.0000 (99.4276)
Valid: 31 [ 150/390]  Loss: 0.5172 (0.447)  Acc@1: 89.0625 (86.5791)  Acc@5: 100.0000 (99.4102)
Valid: 31 [ 200/390]  Loss: 0.4030 (0.447)  Acc@1: 82.8125 (86.5127)  Acc@5: 100.0000 (99.4014)
Valid: 31 [ 250/390]  Loss: 0.4324 (0.453)  Acc@1: 82.8125 (86.2674)  Acc@5: 100.0000 (99.3650)
Valid: 31 [ 300/390]  Loss: 0.3228 (0.454)  Acc@1: 85.9375 (86.1296)  Acc@5: 100.0000 (99.3823)
Valid: 31 [ 350/390]  Loss: 0.2582 (0.453)  Acc@1: 89.0625 (86.1423)  Acc@5: 100.0000 (99.3812)
Valid: 31 [ 390/390]  Loss: 0.2738 (0.453)  Acc@1: 87.5000 (86.0960)  Acc@5: 97.5000 (99.3840)
valid_acc 86.096000
epoch = 31   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2786, 0.7214],
        [0.2798, 0.7202],
        [0.2115, 0.7885],
        [0.2902, 0.7098],
        [0.3442, 0.6558],
        [0.3650, 0.6350],
        [0.4038, 0.5962],
        [0.4835, 0.5165],
        [0.3612, 0.6388],
        [0.2573, 0.7427],
        [0.4810, 0.5190],
        [0.3386, 0.6614],
        [0.4002, 0.5998],
        [0.1568, 0.8432]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4019, 0.5981],
        [0.3649, 0.6351],
        [0.4315, 0.5685],
        [0.2986, 0.7014],
        [0.3960, 0.6040],
        [0.3413, 0.6587],
        [0.3632, 0.6368],
        [0.3622, 0.6378],
        [0.3100, 0.6900],
        [0.3296, 0.6704],
        [0.3834, 0.6166],
        [0.3521, 0.6479],
        [0.3017, 0.6983],
        [0.3980, 0.6020]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 32 [   0/390]  Loss: 0.1174 (0.117)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 7.891e-03
Train: 32 [  50/390]  Loss: 0.1862 (0.165)  Acc@1: 92.1875 (94.2402)  Acc@5: 100.0000 (99.9694)LR: 7.891e-03
Train: 32 [ 100/390]  Loss: 0.2067 (0.173)  Acc@1: 92.1875 (94.1058)  Acc@5: 100.0000 (99.8917)LR: 7.891e-03
Train: 32 [ 150/390]  Loss: 0.08888 (0.175)  Acc@1: 96.8750 (94.1018)  Acc@5: 100.0000 (99.8965)LR: 7.891e-03
Train: 32 [ 200/390]  Loss: 0.1572 (0.175)  Acc@1: 96.8750 (94.0143)  Acc@5: 100.0000 (99.9067)LR: 7.891e-03
Train: 32 [ 250/390]  Loss: 0.2855 (0.176)  Acc@1: 90.6250 (93.9554)  Acc@5: 100.0000 (99.9191)LR: 7.891e-03
Train: 32 [ 300/390]  Loss: 0.1242 (0.181)  Acc@1: 95.3125 (93.6825)  Acc@5: 100.0000 (99.9221)LR: 7.891e-03
Train: 32 [ 350/390]  Loss: 0.2211 (0.184)  Acc@1: 92.1875 (93.5675)  Acc@5: 100.0000 (99.9332)LR: 7.891e-03
Train: 32 [ 390/390]  Loss: 0.2116 (0.185)  Acc@1: 95.0000 (93.4760)  Acc@5: 100.0000 (99.9160)LR: 7.891e-03
train_acc 93.476000
Valid: 32 [   0/390]  Loss: 0.4961 (0.496)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 32 [  50/390]  Loss: 0.3267 (0.459)  Acc@1: 90.6250 (86.3051)  Acc@5: 100.0000 (99.2953)
Valid: 32 [ 100/390]  Loss: 0.3708 (0.457)  Acc@1: 85.9375 (86.3088)  Acc@5: 100.0000 (99.3193)
Valid: 32 [ 150/390]  Loss: 0.6389 (0.451)  Acc@1: 81.2500 (86.3825)  Acc@5: 100.0000 (99.3791)
Valid: 32 [ 200/390]  Loss: 0.5894 (0.457)  Acc@1: 75.0000 (86.2640)  Acc@5: 100.0000 (99.3937)
Valid: 32 [ 250/390]  Loss: 0.8134 (0.461)  Acc@1: 78.1250 (86.1803)  Acc@5: 96.8750 (99.3650)
Valid: 32 [ 300/390]  Loss: 0.5971 (0.455)  Acc@1: 85.9375 (86.3320)  Acc@5: 98.4375 (99.3875)
Valid: 32 [ 350/390]  Loss: 0.7390 (0.450)  Acc@1: 85.9375 (86.3960)  Acc@5: 96.8750 (99.4035)
Valid: 32 [ 390/390]  Loss: 0.5655 (0.443)  Acc@1: 77.5000 (86.5240)  Acc@5: 97.5000 (99.3760)
valid_acc 86.524000
epoch = 32   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2744, 0.7256],
        [0.2761, 0.7239],
        [0.2045, 0.7955],
        [0.2866, 0.7134],
        [0.3416, 0.6584],
        [0.3626, 0.6374],
        [0.4028, 0.5972],
        [0.4872, 0.5128],
        [0.3615, 0.6385],
        [0.2587, 0.7413],
        [0.4899, 0.5101],
        [0.3432, 0.6568],
        [0.4081, 0.5919],
        [0.1542, 0.8458]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4045, 0.5955],
        [0.3674, 0.6326],
        [0.4286, 0.5714],
        [0.2965, 0.7035],
        [0.3915, 0.6085],
        [0.3408, 0.6592],
        [0.3659, 0.6341],
        [0.3570, 0.6430],
        [0.3066, 0.6934],
        [0.3262, 0.6738],
        [0.3728, 0.6272],
        [0.3463, 0.6537],
        [0.2954, 0.7046],
        [0.3938, 0.6062]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 33 [   0/390]  Loss: 0.1686 (0.169)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 7.219e-03
Train: 33 [  50/390]  Loss: 0.1969 (0.160)  Acc@1: 93.7500 (94.5159)  Acc@5: 98.4375 (99.9694)LR: 7.219e-03
Train: 33 [ 100/390]  Loss: 0.1791 (0.168)  Acc@1: 92.1875 (94.0439)  Acc@5: 100.0000 (99.9691)LR: 7.219e-03
Train: 33 [ 150/390]  Loss: 0.1888 (0.167)  Acc@1: 93.7500 (94.1432)  Acc@5: 100.0000 (99.9586)LR: 7.219e-03
Train: 33 [ 200/390]  Loss: 0.1570 (0.169)  Acc@1: 92.1875 (94.1465)  Acc@5: 100.0000 (99.9611)LR: 7.219e-03
Train: 33 [ 250/390]  Loss: 0.2388 (0.171)  Acc@1: 93.7500 (94.0301)  Acc@5: 100.0000 (99.9564)LR: 7.219e-03
Train: 33 [ 300/390]  Loss: 0.1053 (0.174)  Acc@1: 93.7500 (93.9317)  Acc@5: 100.0000 (99.9533)LR: 7.219e-03
Train: 33 [ 350/390]  Loss: 0.1845 (0.175)  Acc@1: 90.6250 (93.8835)  Acc@5: 100.0000 (99.9555)LR: 7.219e-03
Train: 33 [ 390/390]  Loss: 0.1410 (0.175)  Acc@1: 95.0000 (93.8600)  Acc@5: 100.0000 (99.9520)LR: 7.219e-03
train_acc 93.860000
Valid: 33 [   0/390]  Loss: 0.1760 (0.176)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)
Valid: 33 [  50/390]  Loss: 0.3998 (0.441)  Acc@1: 87.5000 (86.8873)  Acc@5: 100.0000 (99.3566)
Valid: 33 [ 100/390]  Loss: 0.3221 (0.448)  Acc@1: 87.5000 (86.2933)  Acc@5: 100.0000 (99.4121)
Valid: 33 [ 150/390]  Loss: 0.5017 (0.454)  Acc@1: 79.6875 (86.1651)  Acc@5: 100.0000 (99.3584)
Valid: 33 [ 200/390]  Loss: 0.2352 (0.442)  Acc@1: 92.1875 (86.4583)  Acc@5: 100.0000 (99.3626)
Valid: 33 [ 250/390]  Loss: 0.3994 (0.440)  Acc@1: 93.7500 (86.5289)  Acc@5: 98.4375 (99.3713)
Valid: 33 [ 300/390]  Loss: 0.3435 (0.439)  Acc@1: 90.6250 (86.7836)  Acc@5: 100.0000 (99.3563)
Valid: 33 [ 350/390]  Loss: 0.1508 (0.436)  Acc@1: 92.1875 (86.7833)  Acc@5: 100.0000 (99.3812)
Valid: 33 [ 390/390]  Loss: 0.5504 (0.439)  Acc@1: 82.5000 (86.7000)  Acc@5: 100.0000 (99.3600)
valid_acc 86.700000
epoch = 33   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2693, 0.7307],
        [0.2685, 0.7315],
        [0.1975, 0.8025],
        [0.2843, 0.7157],
        [0.3377, 0.6623],
        [0.3665, 0.6335],
        [0.4022, 0.5978],
        [0.4930, 0.5070],
        [0.3676, 0.6324],
        [0.2567, 0.7433],
        [0.4964, 0.5036],
        [0.3460, 0.6540],
        [0.4135, 0.5865],
        [0.1495, 0.8505]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4038, 0.5962],
        [0.3639, 0.6361],
        [0.4233, 0.5767],
        [0.2888, 0.7112],
        [0.3939, 0.6061],
        [0.3398, 0.6602],
        [0.3565, 0.6435],
        [0.3542, 0.6458],
        [0.3017, 0.6983],
        [0.3211, 0.6789],
        [0.3671, 0.6329],
        [0.3433, 0.6567],
        [0.2909, 0.7091],
        [0.3904, 0.6096]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 34 [   0/390]  Loss: 0.1156 (0.116)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 6.570e-03
Train: 34 [  50/390]  Loss: 0.1296 (0.126)  Acc@1: 96.8750 (95.9559)  Acc@5: 100.0000 (99.8775)LR: 6.570e-03
Train: 34 [ 100/390]  Loss: 0.1250 (0.136)  Acc@1: 95.3125 (95.1578)  Acc@5: 100.0000 (99.9381)LR: 6.570e-03
Train: 34 [ 150/390]  Loss: 0.1964 (0.142)  Acc@1: 92.1875 (94.8986)  Acc@5: 100.0000 (99.9379)LR: 6.570e-03
Train: 34 [ 200/390]  Loss: 0.1676 (0.142)  Acc@1: 95.3125 (94.8228)  Acc@5: 100.0000 (99.9456)LR: 6.570e-03
Train: 34 [ 250/390]  Loss: 0.2322 (0.145)  Acc@1: 92.1875 (94.7273)  Acc@5: 100.0000 (99.9315)LR: 6.570e-03
Train: 34 [ 300/390]  Loss: 0.09652 (0.150)  Acc@1: 95.3125 (94.6117)  Acc@5: 100.0000 (99.9429)LR: 6.570e-03
Train: 34 [ 350/390]  Loss: 0.3151 (0.151)  Acc@1: 87.5000 (94.5379)  Acc@5: 100.0000 (99.9377)LR: 6.570e-03
Train: 34 [ 390/390]  Loss: 0.1631 (0.154)  Acc@1: 92.5000 (94.4720)  Acc@5: 100.0000 (99.9320)LR: 6.570e-03
train_acc 94.472000
Valid: 34 [   0/390]  Loss: 0.3555 (0.356)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 34 [  50/390]  Loss: 0.3514 (0.443)  Acc@1: 89.0625 (86.9485)  Acc@5: 100.0000 (99.3566)
Valid: 34 [ 100/390]  Loss: 0.2190 (0.445)  Acc@1: 92.1875 (87.2061)  Acc@5: 100.0000 (99.3502)
Valid: 34 [ 150/390]  Loss: 0.6168 (0.440)  Acc@1: 84.3750 (87.3655)  Acc@5: 100.0000 (99.4412)
Valid: 34 [ 200/390]  Loss: 0.5282 (0.443)  Acc@1: 84.3750 (87.1891)  Acc@5: 100.0000 (99.4248)
Valid: 34 [ 250/390]  Loss: 0.3132 (0.441)  Acc@1: 92.1875 (87.0393)  Acc@5: 100.0000 (99.3899)
Valid: 34 [ 300/390]  Loss: 0.5599 (0.445)  Acc@1: 89.0625 (86.9342)  Acc@5: 98.4375 (99.3926)
Valid: 34 [ 350/390]  Loss: 0.3136 (0.448)  Acc@1: 90.6250 (86.7388)  Acc@5: 100.0000 (99.4035)
Valid: 34 [ 390/390]  Loss: 0.3592 (0.448)  Acc@1: 85.0000 (86.7280)  Acc@5: 100.0000 (99.4000)
valid_acc 86.728000
epoch = 34   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2663, 0.7337],
        [0.2593, 0.7407],
        [0.1912, 0.8088],
        [0.2779, 0.7221],
        [0.3334, 0.6666],
        [0.3658, 0.6342],
        [0.4042, 0.5958],
        [0.4977, 0.5023],
        [0.3694, 0.6306],
        [0.2559, 0.7441],
        [0.5051, 0.4949],
        [0.3539, 0.6461],
        [0.4159, 0.5841],
        [0.1442, 0.8558]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4028, 0.5972],
        [0.3636, 0.6364],
        [0.4227, 0.5773],
        [0.2849, 0.7151],
        [0.3938, 0.6062],
        [0.3324, 0.6676],
        [0.3591, 0.6409],
        [0.3513, 0.6487],
        [0.3001, 0.6999],
        [0.3170, 0.6830],
        [0.3636, 0.6364],
        [0.3366, 0.6634],
        [0.2822, 0.7178],
        [0.3813, 0.6187]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 35 [   0/390]  Loss: 0.07815 (0.0781)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 5.947e-03
Train: 35 [  50/390]  Loss: 0.1273 (0.140)  Acc@1: 93.7500 (95.2206)  Acc@5: 100.0000 (100.0000)LR: 5.947e-03
Train: 35 [ 100/390]  Loss: 0.1664 (0.142)  Acc@1: 95.3125 (94.8793)  Acc@5: 100.0000 (100.0000)LR: 5.947e-03
Train: 35 [ 150/390]  Loss: 0.05748 (0.140)  Acc@1: 98.4375 (94.9710)  Acc@5: 100.0000 (99.9897)LR: 5.947e-03
Train: 35 [ 200/390]  Loss: 0.1070 (0.140)  Acc@1: 95.3125 (94.9160)  Acc@5: 100.0000 (99.9767)LR: 5.947e-03
Train: 35 [ 250/390]  Loss: 0.1892 (0.143)  Acc@1: 90.6250 (94.7834)  Acc@5: 100.0000 (99.9689)LR: 5.947e-03
Train: 35 [ 300/390]  Loss: 0.01714 (0.144)  Acc@1: 100.0000 (94.7830)  Acc@5: 100.0000 (99.9637)LR: 5.947e-03
Train: 35 [ 350/390]  Loss: 0.09290 (0.143)  Acc@1: 96.8750 (94.8495)  Acc@5: 100.0000 (99.9644)LR: 5.947e-03
Train: 35 [ 390/390]  Loss: 0.1710 (0.143)  Acc@1: 97.5000 (94.8800)  Acc@5: 97.5000 (99.9640)LR: 5.947e-03
train_acc 94.880000
Valid: 35 [   0/390]  Loss: 0.4445 (0.445)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 35 [  50/390]  Loss: 0.4615 (0.453)  Acc@1: 85.9375 (86.9792)  Acc@5: 100.0000 (99.4179)
Valid: 35 [ 100/390]  Loss: 0.4545 (0.455)  Acc@1: 85.9375 (86.8348)  Acc@5: 100.0000 (99.3812)
Valid: 35 [ 150/390]  Loss: 0.6253 (0.460)  Acc@1: 82.8125 (86.5998)  Acc@5: 98.4375 (99.3998)
Valid: 35 [ 200/390]  Loss: 0.7105 (0.460)  Acc@1: 85.9375 (86.7615)  Acc@5: 98.4375 (99.3859)
Valid: 35 [ 250/390]  Loss: 0.4148 (0.456)  Acc@1: 89.0625 (86.9273)  Acc@5: 98.4375 (99.4086)
Valid: 35 [ 300/390]  Loss: 0.3561 (0.458)  Acc@1: 89.0625 (86.9653)  Acc@5: 100.0000 (99.4498)
Valid: 35 [ 350/390]  Loss: 0.1483 (0.450)  Acc@1: 95.3125 (87.1928)  Acc@5: 100.0000 (99.4302)
Valid: 35 [ 390/390]  Loss: 0.7932 (0.452)  Acc@1: 82.5000 (87.1800)  Acc@5: 100.0000 (99.4080)
valid_acc 87.180000
epoch = 35   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2604, 0.7396],
        [0.2493, 0.7507],
        [0.1838, 0.8162],
        [0.2730, 0.7270],
        [0.3306, 0.6694],
        [0.3602, 0.6398],
        [0.4076, 0.5924],
        [0.5066, 0.4934],
        [0.3680, 0.6320],
        [0.2519, 0.7481],
        [0.5119, 0.4881],
        [0.3581, 0.6419],
        [0.4184, 0.5816],
        [0.1393, 0.8607]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4024, 0.5976],
        [0.3561, 0.6439],
        [0.4230, 0.5770],
        [0.2833, 0.7167],
        [0.3974, 0.6026],
        [0.3293, 0.6707],
        [0.3559, 0.6441],
        [0.3496, 0.6504],
        [0.2962, 0.7038],
        [0.3157, 0.6843],
        [0.3597, 0.6403],
        [0.3367, 0.6633],
        [0.2787, 0.7213],
        [0.3879, 0.6121]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 36 [   0/390]  Loss: 0.1955 (0.196)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 5.351e-03
Train: 36 [  50/390]  Loss: 0.06232 (0.149)  Acc@1: 96.8750 (94.5772)  Acc@5: 100.0000 (99.9081)LR: 5.351e-03
Train: 36 [ 100/390]  Loss: 0.1255 (0.136)  Acc@1: 96.8750 (95.2042)  Acc@5: 100.0000 (99.9381)LR: 5.351e-03
Train: 36 [ 150/390]  Loss: 0.1226 (0.131)  Acc@1: 96.8750 (95.3642)  Acc@5: 100.0000 (99.9483)LR: 5.351e-03
Train: 36 [ 200/390]  Loss: 0.03672 (0.128)  Acc@1: 98.4375 (95.4680)  Acc@5: 100.0000 (99.9611)LR: 5.351e-03
Train: 36 [ 250/390]  Loss: 0.1547 (0.132)  Acc@1: 93.7500 (95.3436)  Acc@5: 100.0000 (99.9564)LR: 5.351e-03
Train: 36 [ 300/390]  Loss: 0.1576 (0.132)  Acc@1: 95.3125 (95.3073)  Acc@5: 100.0000 (99.9585)LR: 5.351e-03
Train: 36 [ 350/390]  Loss: 0.1796 (0.132)  Acc@1: 92.1875 (95.3348)  Acc@5: 100.0000 (99.9555)LR: 5.351e-03
Train: 36 [ 390/390]  Loss: 0.1903 (0.134)  Acc@1: 90.0000 (95.2880)  Acc@5: 100.0000 (99.9520)LR: 5.351e-03
train_acc 95.288000
Valid: 36 [   0/390]  Loss: 0.5057 (0.506)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 36 [  50/390]  Loss: 0.4992 (0.437)  Acc@1: 84.3750 (87.8064)  Acc@5: 100.0000 (99.4179)
Valid: 36 [ 100/390]  Loss: 0.3960 (0.443)  Acc@1: 85.9375 (87.0823)  Acc@5: 100.0000 (99.3502)
Valid: 36 [ 150/390]  Loss: 0.4437 (0.455)  Acc@1: 87.5000 (87.0344)  Acc@5: 100.0000 (99.2860)
Valid: 36 [ 200/390]  Loss: 0.2673 (0.455)  Acc@1: 92.1875 (87.1657)  Acc@5: 100.0000 (99.3004)
Valid: 36 [ 250/390]  Loss: 0.4729 (0.455)  Acc@1: 89.0625 (87.1389)  Acc@5: 100.0000 (99.2592)
Valid: 36 [ 300/390]  Loss: 0.3168 (0.447)  Acc@1: 90.6250 (87.1885)  Acc@5: 100.0000 (99.3148)
Valid: 36 [ 350/390]  Loss: 0.1275 (0.447)  Acc@1: 95.3125 (87.2552)  Acc@5: 100.0000 (99.3545)
Valid: 36 [ 390/390]  Loss: 0.2428 (0.447)  Acc@1: 92.5000 (87.2400)  Acc@5: 100.0000 (99.3720)
valid_acc 87.240000
epoch = 36   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2568, 0.7432],
        [0.2458, 0.7542],
        [0.1786, 0.8214],
        [0.2691, 0.7309],
        [0.3294, 0.6706],
        [0.3605, 0.6395],
        [0.4081, 0.5919],
        [0.5152, 0.4848],
        [0.3684, 0.6316],
        [0.2510, 0.7490],
        [0.5214, 0.4786],
        [0.3650, 0.6350],
        [0.4263, 0.5737],
        [0.1363, 0.8637]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3999, 0.6001],
        [0.3572, 0.6428],
        [0.4203, 0.5797],
        [0.2801, 0.7199],
        [0.3917, 0.6083],
        [0.3193, 0.6807],
        [0.3479, 0.6521],
        [0.3490, 0.6510],
        [0.2914, 0.7086],
        [0.3080, 0.6920],
        [0.3596, 0.6404],
        [0.3332, 0.6668],
        [0.2743, 0.7257],
        [0.3833, 0.6167]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 37 [   0/390]  Loss: 0.1603 (0.160)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 4.785e-03
Train: 37 [  50/390]  Loss: 0.1835 (0.124)  Acc@1: 96.8750 (95.7414)  Acc@5: 96.8750 (99.9081)LR: 4.785e-03
Train: 37 [ 100/390]  Loss: 0.08031 (0.117)  Acc@1: 95.3125 (95.9932)  Acc@5: 100.0000 (99.9536)LR: 4.785e-03
Train: 37 [ 150/390]  Loss: 0.1552 (0.119)  Acc@1: 92.1875 (95.9437)  Acc@5: 100.0000 (99.9690)LR: 4.785e-03
Train: 37 [ 200/390]  Loss: 0.09793 (0.118)  Acc@1: 98.4375 (95.9499)  Acc@5: 100.0000 (99.9767)LR: 4.785e-03
Train: 37 [ 250/390]  Loss: 0.09310 (0.118)  Acc@1: 95.3125 (95.8852)  Acc@5: 100.0000 (99.9751)LR: 4.785e-03
Train: 37 [ 300/390]  Loss: 0.1106 (0.119)  Acc@1: 96.8750 (95.8679)  Acc@5: 100.0000 (99.9792)LR: 4.785e-03
Train: 37 [ 350/390]  Loss: 0.2306 (0.120)  Acc@1: 95.3125 (95.7666)  Acc@5: 100.0000 (99.9777)LR: 4.785e-03
Train: 37 [ 390/390]  Loss: 0.2453 (0.121)  Acc@1: 95.0000 (95.7720)  Acc@5: 100.0000 (99.9720)LR: 4.785e-03
train_acc 95.772000
Valid: 37 [   0/390]  Loss: 0.6612 (0.661)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 37 [  50/390]  Loss: 0.5134 (0.452)  Acc@1: 84.3750 (86.9792)  Acc@5: 98.4375 (99.4179)
Valid: 37 [ 100/390]  Loss: 0.4550 (0.436)  Acc@1: 85.9375 (87.2525)  Acc@5: 98.4375 (99.4276)
Valid: 37 [ 150/390]  Loss: 0.4997 (0.435)  Acc@1: 85.9375 (87.3137)  Acc@5: 100.0000 (99.4723)
Valid: 37 [ 200/390]  Loss: 0.5820 (0.436)  Acc@1: 79.6875 (87.4845)  Acc@5: 100.0000 (99.5025)
Valid: 37 [ 250/390]  Loss: 0.4007 (0.431)  Acc@1: 89.0625 (87.5374)  Acc@5: 100.0000 (99.5082)
Valid: 37 [ 300/390]  Loss: 0.1970 (0.432)  Acc@1: 96.8750 (87.5934)  Acc@5: 100.0000 (99.5172)
Valid: 37 [ 350/390]  Loss: 0.5144 (0.442)  Acc@1: 85.9375 (87.4065)  Acc@5: 100.0000 (99.4925)
Valid: 37 [ 390/390]  Loss: 0.2591 (0.441)  Acc@1: 90.0000 (87.4560)  Acc@5: 100.0000 (99.4920)
valid_acc 87.456000
epoch = 37   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2473, 0.7527],
        [0.2414, 0.7586],
        [0.1723, 0.8277],
        [0.2678, 0.7322],
        [0.3296, 0.6704],
        [0.3650, 0.6350],
        [0.4103, 0.5897],
        [0.5242, 0.4758],
        [0.3776, 0.6224],
        [0.2546, 0.7454],
        [0.5316, 0.4684],
        [0.3741, 0.6259],
        [0.4356, 0.5644],
        [0.1349, 0.8651]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3947, 0.6053],
        [0.3533, 0.6467],
        [0.4150, 0.5850],
        [0.2765, 0.7235],
        [0.3844, 0.6156],
        [0.3159, 0.6841],
        [0.3485, 0.6515],
        [0.3452, 0.6548],
        [0.2919, 0.7081],
        [0.3024, 0.6976],
        [0.3560, 0.6440],
        [0.3293, 0.6707],
        [0.2711, 0.7289],
        [0.3811, 0.6189]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 38 [   0/390]  Loss: 0.1155 (0.115)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 4.252e-03
Train: 38 [  50/390]  Loss: 0.1135 (0.109)  Acc@1: 96.8750 (96.4154)  Acc@5: 100.0000 (100.0000)LR: 4.252e-03
Train: 38 [ 100/390]  Loss: 0.1008 (0.107)  Acc@1: 95.3125 (96.5501)  Acc@5: 100.0000 (99.9536)LR: 4.252e-03
Train: 38 [ 150/390]  Loss: 0.04743 (0.105)  Acc@1: 96.8750 (96.5025)  Acc@5: 100.0000 (99.9379)LR: 4.252e-03
Train: 38 [ 200/390]  Loss: 0.05561 (0.106)  Acc@1: 98.4375 (96.3853)  Acc@5: 100.0000 (99.9456)LR: 4.252e-03
Train: 38 [ 250/390]  Loss: 0.07004 (0.108)  Acc@1: 98.4375 (96.3272)  Acc@5: 100.0000 (99.9502)LR: 4.252e-03
Train: 38 [ 300/390]  Loss: 0.1244 (0.111)  Acc@1: 95.3125 (96.2313)  Acc@5: 100.0000 (99.9481)LR: 4.252e-03
Train: 38 [ 350/390]  Loss: 0.1020 (0.112)  Acc@1: 93.7500 (96.1627)  Acc@5: 100.0000 (99.9510)LR: 4.252e-03
Train: 38 [ 390/390]  Loss: 0.1218 (0.112)  Acc@1: 95.0000 (96.1720)  Acc@5: 100.0000 (99.9520)LR: 4.252e-03
train_acc 96.172000
Valid: 38 [   0/390]  Loss: 0.3082 (0.308)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 38 [  50/390]  Loss: 0.6227 (0.497)  Acc@1: 79.6875 (87.2243)  Acc@5: 98.4375 (99.4179)
Valid: 38 [ 100/390]  Loss: 0.3036 (0.458)  Acc@1: 90.6250 (87.4691)  Acc@5: 100.0000 (99.5204)
Valid: 38 [ 150/390]  Loss: 0.4320 (0.443)  Acc@1: 82.8125 (87.7690)  Acc@5: 100.0000 (99.5447)
Valid: 38 [ 200/390]  Loss: 0.3021 (0.444)  Acc@1: 89.0625 (87.5622)  Acc@5: 100.0000 (99.4947)
Valid: 38 [ 250/390]  Loss: 0.5687 (0.448)  Acc@1: 85.9375 (87.5934)  Acc@5: 100.0000 (99.5082)
Valid: 38 [ 300/390]  Loss: 0.4927 (0.447)  Acc@1: 89.0625 (87.6402)  Acc@5: 96.8750 (99.4446)
Valid: 38 [ 350/390]  Loss: 0.3984 (0.445)  Acc@1: 84.3750 (87.6113)  Acc@5: 100.0000 (99.4480)
Valid: 38 [ 390/390]  Loss: 0.7421 (0.441)  Acc@1: 85.0000 (87.6640)  Acc@5: 97.5000 (99.4680)
valid_acc 87.664000
epoch = 38   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2355, 0.7645],
        [0.2380, 0.7620],
        [0.1621, 0.8379],
        [0.2665, 0.7335],
        [0.3337, 0.6663],
        [0.3648, 0.6352],
        [0.4117, 0.5883],
        [0.5329, 0.4671],
        [0.3789, 0.6211],
        [0.2504, 0.7496],
        [0.5409, 0.4591],
        [0.3789, 0.6211],
        [0.4390, 0.5610],
        [0.1327, 0.8673]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3952, 0.6048],
        [0.3482, 0.6518],
        [0.4128, 0.5872],
        [0.2703, 0.7297],
        [0.3815, 0.6185],
        [0.3116, 0.6884],
        [0.3474, 0.6526],
        [0.3412, 0.6588],
        [0.2922, 0.7078],
        [0.2957, 0.7043],
        [0.3523, 0.6477],
        [0.3275, 0.6725],
        [0.2667, 0.7333],
        [0.3762, 0.6238]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 39 [   0/390]  Loss: 0.09927 (0.0993)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [  50/390]  Loss: 0.05292 (0.0917)  Acc@1: 96.8750 (96.8444)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [ 100/390]  Loss: 0.2373 (0.0930)  Acc@1: 93.7500 (96.7667)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [ 150/390]  Loss: 0.06905 (0.0974)  Acc@1: 96.8750 (96.5335)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [ 200/390]  Loss: 0.06162 (0.0989)  Acc@1: 98.4375 (96.4475)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [ 250/390]  Loss: 0.1854 (0.0972)  Acc@1: 93.7500 (96.5015)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [ 300/390]  Loss: 0.06920 (0.0981)  Acc@1: 98.4375 (96.4286)  Acc@5: 100.0000 (99.9948)LR: 3.754e-03
Train: 39 [ 350/390]  Loss: 0.2745 (0.101)  Acc@1: 90.6250 (96.3853)  Acc@5: 98.4375 (99.9866)LR: 3.754e-03
Train: 39 [ 390/390]  Loss: 0.2373 (0.0994)  Acc@1: 92.5000 (96.4080)  Acc@5: 100.0000 (99.9840)LR: 3.754e-03
train_acc 96.408000
Valid: 39 [   0/390]  Loss: 0.5862 (0.586)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 39 [  50/390]  Loss: 0.1263 (0.437)  Acc@1: 95.3125 (87.9596)  Acc@5: 100.0000 (99.4792)
Valid: 39 [ 100/390]  Loss: 0.3161 (0.440)  Acc@1: 93.7500 (87.7321)  Acc@5: 100.0000 (99.4276)
Valid: 39 [ 150/390]  Loss: 0.2709 (0.433)  Acc@1: 89.0625 (87.8829)  Acc@5: 100.0000 (99.5447)
Valid: 39 [ 200/390]  Loss: 0.3558 (0.447)  Acc@1: 90.6250 (87.4067)  Acc@5: 100.0000 (99.5025)
Valid: 39 [ 250/390]  Loss: 0.6343 (0.456)  Acc@1: 82.8125 (87.3942)  Acc@5: 98.4375 (99.4895)
Valid: 39 [ 300/390]  Loss: 0.8739 (0.462)  Acc@1: 79.6875 (87.4014)  Acc@5: 98.4375 (99.4290)
Valid: 39 [ 350/390]  Loss: 0.5325 (0.454)  Acc@1: 84.3750 (87.5534)  Acc@5: 100.0000 (99.4525)
Valid: 39 [ 390/390]  Loss: 0.9140 (0.451)  Acc@1: 85.0000 (87.6160)  Acc@5: 97.5000 (99.4440)
valid_acc 87.616000
epoch = 39   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2304, 0.7696],
        [0.2283, 0.7717],
        [0.1565, 0.8435],
        [0.2612, 0.7388],
        [0.3307, 0.6693],
        [0.3624, 0.6376],
        [0.4111, 0.5889],
        [0.5352, 0.4648],
        [0.3819, 0.6181],
        [0.2448, 0.7552],
        [0.5515, 0.4485],
        [0.3813, 0.6187],
        [0.4425, 0.5575],
        [0.1294, 0.8706]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3934, 0.6066],
        [0.3425, 0.6575],
        [0.4133, 0.5867],
        [0.2656, 0.7344],
        [0.3821, 0.6179],
        [0.3042, 0.6958],
        [0.3403, 0.6597],
        [0.3419, 0.6581],
        [0.2893, 0.7107],
        [0.2920, 0.7080],
        [0.3519, 0.6481],
        [0.3230, 0.6770],
        [0.2636, 0.7364],
        [0.3771, 0.6229]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 40 [   0/390]  Loss: 0.09067 (0.0907)  Acc@1: 98.4375 (98.4375)  Acc@5: 98.4375 (98.4375)LR: 3.292e-03
Train: 40 [  50/390]  Loss: 0.1462 (0.0970)  Acc@1: 93.7500 (96.5993)  Acc@5: 100.0000 (99.9387)LR: 3.292e-03
Train: 40 [ 100/390]  Loss: 0.03133 (0.0938)  Acc@1: 98.4375 (96.8131)  Acc@5: 100.0000 (99.9536)LR: 3.292e-03
Train: 40 [ 150/390]  Loss: 0.04802 (0.0872)  Acc@1: 100.0000 (97.0613)  Acc@5: 100.0000 (99.9690)LR: 3.292e-03
Train: 40 [ 200/390]  Loss: 0.04735 (0.0876)  Acc@1: 98.4375 (97.1004)  Acc@5: 100.0000 (99.9767)LR: 3.292e-03
Train: 40 [ 250/390]  Loss: 0.2562 (0.0887)  Acc@1: 87.5000 (97.0244)  Acc@5: 100.0000 (99.9813)LR: 3.292e-03
Train: 40 [ 300/390]  Loss: 0.07664 (0.0884)  Acc@1: 96.8750 (97.0203)  Acc@5: 100.0000 (99.9844)LR: 3.292e-03
Train: 40 [ 350/390]  Loss: 0.09982 (0.0892)  Acc@1: 96.8750 (96.9863)  Acc@5: 100.0000 (99.9866)LR: 3.292e-03
Train: 40 [ 390/390]  Loss: 0.1003 (0.0915)  Acc@1: 97.5000 (96.9160)  Acc@5: 100.0000 (99.9880)LR: 3.292e-03
train_acc 96.916000
Valid: 40 [   0/390]  Loss: 0.4682 (0.468)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 40 [  50/390]  Loss: 0.4973 (0.444)  Acc@1: 82.8125 (87.4387)  Acc@5: 100.0000 (99.5404)
Valid: 40 [ 100/390]  Loss: 0.4157 (0.440)  Acc@1: 85.9375 (87.8403)  Acc@5: 98.4375 (99.4121)
Valid: 40 [ 150/390]  Loss: 0.2851 (0.433)  Acc@1: 90.6250 (87.9450)  Acc@5: 100.0000 (99.4723)
Valid: 40 [ 200/390]  Loss: 0.2283 (0.450)  Acc@1: 90.6250 (87.4922)  Acc@5: 100.0000 (99.4092)
Valid: 40 [ 250/390]  Loss: 0.2200 (0.440)  Acc@1: 92.1875 (87.8113)  Acc@5: 100.0000 (99.4646)
Valid: 40 [ 300/390]  Loss: 0.3086 (0.442)  Acc@1: 92.1875 (87.8115)  Acc@5: 98.4375 (99.4861)
Valid: 40 [ 350/390]  Loss: 0.2930 (0.445)  Acc@1: 90.6250 (87.7226)  Acc@5: 100.0000 (99.5103)
Valid: 40 [ 390/390]  Loss: 0.4056 (0.450)  Acc@1: 90.0000 (87.6320)  Acc@5: 100.0000 (99.4640)
valid_acc 87.632000
epoch = 40   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2247, 0.7753],
        [0.2267, 0.7733],
        [0.1511, 0.8489],
        [0.2584, 0.7416],
        [0.3311, 0.6689],
        [0.3643, 0.6357],
        [0.4103, 0.5897],
        [0.5411, 0.4589],
        [0.3883, 0.6117],
        [0.2432, 0.7568],
        [0.5621, 0.4379],
        [0.3842, 0.6158],
        [0.4492, 0.5508],
        [0.1265, 0.8735]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3988, 0.6012],
        [0.3427, 0.6573],
        [0.4113, 0.5887],
        [0.2612, 0.7388],
        [0.3740, 0.6260],
        [0.3018, 0.6982],
        [0.3356, 0.6644],
        [0.3391, 0.6609],
        [0.2846, 0.7154],
        [0.2911, 0.7089],
        [0.3466, 0.6534],
        [0.3175, 0.6825],
        [0.2580, 0.7420],
        [0.3776, 0.6224]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 41 [   0/390]  Loss: 0.1556 (0.156)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 2.868e-03
Train: 41 [  50/390]  Loss: 0.1175 (0.0827)  Acc@1: 95.3125 (97.1814)  Acc@5: 100.0000 (99.9694)LR: 2.868e-03
Train: 41 [ 100/390]  Loss: 0.05954 (0.0869)  Acc@1: 98.4375 (96.9369)  Acc@5: 100.0000 (99.9845)LR: 2.868e-03
Train: 41 [ 150/390]  Loss: 0.1120 (0.0855)  Acc@1: 96.8750 (96.9992)  Acc@5: 100.0000 (99.9897)LR: 2.868e-03
Train: 41 [ 200/390]  Loss: 0.07791 (0.0841)  Acc@1: 98.4375 (97.0927)  Acc@5: 100.0000 (99.9922)LR: 2.868e-03
Train: 41 [ 250/390]  Loss: 0.03720 (0.0843)  Acc@1: 100.0000 (97.0742)  Acc@5: 100.0000 (99.9938)LR: 2.868e-03
Train: 41 [ 300/390]  Loss: 0.06698 (0.0850)  Acc@1: 98.4375 (97.0411)  Acc@5: 100.0000 (99.9948)LR: 2.868e-03
Train: 41 [ 350/390]  Loss: 0.06241 (0.0856)  Acc@1: 96.8750 (97.0264)  Acc@5: 100.0000 (99.9955)LR: 2.868e-03
Train: 41 [ 390/390]  Loss: 0.1610 (0.0870)  Acc@1: 95.0000 (96.9640)  Acc@5: 100.0000 (99.9920)LR: 2.868e-03
train_acc 96.964000
Valid: 41 [   0/390]  Loss: 0.5334 (0.533)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)
Valid: 41 [  50/390]  Loss: 0.6123 (0.474)  Acc@1: 85.9375 (87.1936)  Acc@5: 100.0000 (99.4792)
Valid: 41 [ 100/390]  Loss: 0.5423 (0.479)  Acc@1: 79.6875 (87.1751)  Acc@5: 98.4375 (99.4121)
Valid: 41 [ 150/390]  Loss: 0.5166 (0.471)  Acc@1: 81.2500 (87.4276)  Acc@5: 100.0000 (99.4412)
Valid: 41 [ 200/390]  Loss: 0.3985 (0.463)  Acc@1: 89.0625 (87.6244)  Acc@5: 100.0000 (99.4481)
Valid: 41 [ 250/390]  Loss: 0.2434 (0.455)  Acc@1: 92.1875 (87.6494)  Acc@5: 100.0000 (99.4895)
Valid: 41 [ 300/390]  Loss: 0.4605 (0.454)  Acc@1: 85.9375 (87.6453)  Acc@5: 98.4375 (99.4653)
Valid: 41 [ 350/390]  Loss: 0.3642 (0.459)  Acc@1: 84.3750 (87.5623)  Acc@5: 100.0000 (99.4881)
Valid: 41 [ 390/390]  Loss: 0.4134 (0.461)  Acc@1: 87.5000 (87.5480)  Acc@5: 100.0000 (99.4480)
valid_acc 87.548000
epoch = 41   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2203, 0.7797],
        [0.2248, 0.7752],
        [0.1459, 0.8541],
        [0.2567, 0.7433],
        [0.3260, 0.6740],
        [0.3651, 0.6349],
        [0.4116, 0.5884],
        [0.5481, 0.4519],
        [0.3934, 0.6066],
        [0.2416, 0.7584],
        [0.5759, 0.4241],
        [0.3883, 0.6117],
        [0.4592, 0.5408],
        [0.1232, 0.8768]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3998, 0.6002],
        [0.3402, 0.6598],
        [0.4081, 0.5919],
        [0.2569, 0.7431],
        [0.3692, 0.6308],
        [0.2982, 0.7018],
        [0.3314, 0.6686],
        [0.3366, 0.6634],
        [0.2814, 0.7186],
        [0.2876, 0.7124],
        [0.3475, 0.6525],
        [0.3135, 0.6865],
        [0.2536, 0.7464],
        [0.3731, 0.6269]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 42 [   0/390]  Loss: 0.1370 (0.137)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [  50/390]  Loss: 0.04850 (0.0652)  Acc@1: 100.0000 (97.9779)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [ 100/390]  Loss: 0.03982 (0.0723)  Acc@1: 100.0000 (97.6485)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [ 150/390]  Loss: 0.02199 (0.0716)  Acc@1: 100.0000 (97.5890)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [ 200/390]  Loss: 0.1270 (0.0757)  Acc@1: 95.3125 (97.3881)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [ 250/390]  Loss: 0.06852 (0.0774)  Acc@1: 96.8750 (97.3170)  Acc@5: 100.0000 (99.9938)LR: 2.484e-03
Train: 42 [ 300/390]  Loss: 0.05892 (0.0769)  Acc@1: 98.4375 (97.3110)  Acc@5: 100.0000 (99.9948)LR: 2.484e-03
Train: 42 [ 350/390]  Loss: 0.04794 (0.0767)  Acc@1: 98.4375 (97.3157)  Acc@5: 100.0000 (99.9911)LR: 2.484e-03
Train: 42 [ 390/390]  Loss: 0.04448 (0.0770)  Acc@1: 100.0000 (97.3160)  Acc@5: 100.0000 (99.9920)LR: 2.484e-03
train_acc 97.316000
Valid: 42 [   0/390]  Loss: 0.3249 (0.325)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 42 [  50/390]  Loss: 0.5703 (0.413)  Acc@1: 87.5000 (88.7868)  Acc@5: 100.0000 (99.7243)
Valid: 42 [ 100/390]  Loss: 0.5872 (0.430)  Acc@1: 82.8125 (88.3973)  Acc@5: 100.0000 (99.5514)
Valid: 42 [ 150/390]  Loss: 0.4887 (0.436)  Acc@1: 84.3750 (88.4002)  Acc@5: 98.4375 (99.5137)
Valid: 42 [ 200/390]  Loss: 0.6973 (0.444)  Acc@1: 82.8125 (88.1141)  Acc@5: 96.8750 (99.4714)
Valid: 42 [ 250/390]  Loss: 0.5372 (0.451)  Acc@1: 90.6250 (87.9482)  Acc@5: 100.0000 (99.4584)
Valid: 42 [ 300/390]  Loss: 0.3213 (0.450)  Acc@1: 90.6250 (88.0035)  Acc@5: 98.4375 (99.4757)
Valid: 42 [ 350/390]  Loss: 0.2862 (0.445)  Acc@1: 93.7500 (88.1366)  Acc@5: 100.0000 (99.4881)
Valid: 42 [ 390/390]  Loss: 0.5116 (0.443)  Acc@1: 85.0000 (88.1160)  Acc@5: 100.0000 (99.4880)
valid_acc 88.116000
epoch = 42   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2150, 0.7850],
        [0.2196, 0.7804],
        [0.1413, 0.8587],
        [0.2559, 0.7441],
        [0.3272, 0.6728],
        [0.3669, 0.6331],
        [0.4142, 0.5858],
        [0.5527, 0.4473],
        [0.4024, 0.5976],
        [0.2409, 0.7591],
        [0.5864, 0.4136],
        [0.3989, 0.6011],
        [0.4685, 0.5315],
        [0.1229, 0.8771]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3986, 0.6014],
        [0.3358, 0.6642],
        [0.4022, 0.5978],
        [0.2558, 0.7442],
        [0.3677, 0.6323],
        [0.2909, 0.7091],
        [0.3278, 0.6722],
        [0.3330, 0.6670],
        [0.2784, 0.7216],
        [0.2874, 0.7126],
        [0.3474, 0.6526],
        [0.3106, 0.6894],
        [0.2479, 0.7521],
        [0.3733, 0.6267]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 43 [   0/390]  Loss: 0.07283 (0.0728)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [  50/390]  Loss: 0.05635 (0.0695)  Acc@1: 96.8750 (97.7022)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [ 100/390]  Loss: 0.04199 (0.0727)  Acc@1: 98.4375 (97.6021)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [ 150/390]  Loss: 0.08984 (0.0767)  Acc@1: 95.3125 (97.3924)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [ 200/390]  Loss: 0.03891 (0.0730)  Acc@1: 100.0000 (97.5669)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [ 250/390]  Loss: 0.1103 (0.0718)  Acc@1: 95.3125 (97.6033)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [ 300/390]  Loss: 0.05957 (0.0710)  Acc@1: 96.8750 (97.5602)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [ 350/390]  Loss: 0.09841 (0.0719)  Acc@1: 96.8750 (97.5383)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [ 390/390]  Loss: 0.1781 (0.0717)  Acc@1: 90.0000 (97.5320)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
train_acc 97.532000
Valid: 43 [   0/390]  Loss: 0.5850 (0.585)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 43 [  50/390]  Loss: 0.2382 (0.463)  Acc@1: 92.1875 (87.7145)  Acc@5: 100.0000 (99.4179)
Valid: 43 [ 100/390]  Loss: 0.4643 (0.470)  Acc@1: 89.0625 (87.7630)  Acc@5: 100.0000 (99.4585)
Valid: 43 [ 150/390]  Loss: 0.4361 (0.454)  Acc@1: 87.5000 (87.9450)  Acc@5: 100.0000 (99.5240)
Valid: 43 [ 200/390]  Loss: 0.5326 (0.461)  Acc@1: 89.0625 (87.8032)  Acc@5: 100.0000 (99.4869)
Valid: 43 [ 250/390]  Loss: 0.06896 (0.457)  Acc@1: 96.8750 (87.8735)  Acc@5: 100.0000 (99.4771)
Valid: 43 [ 300/390]  Loss: 0.7404 (0.450)  Acc@1: 81.2500 (88.0035)  Acc@5: 95.3125 (99.4757)
Valid: 43 [ 350/390]  Loss: 0.4755 (0.451)  Acc@1: 79.6875 (87.9363)  Acc@5: 100.0000 (99.5059)
Valid: 43 [ 390/390]  Loss: 0.3400 (0.448)  Acc@1: 92.5000 (87.9800)  Acc@5: 100.0000 (99.5080)
valid_acc 87.980000
epoch = 43   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2100, 0.7900],
        [0.2162, 0.7838],
        [0.1359, 0.8641],
        [0.2515, 0.7485],
        [0.3266, 0.6734],
        [0.3681, 0.6319],
        [0.4117, 0.5883],
        [0.5576, 0.4424],
        [0.4123, 0.5877],
        [0.2355, 0.7645],
        [0.5938, 0.4062],
        [0.4026, 0.5974],
        [0.4790, 0.5210],
        [0.1209, 0.8791]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4014, 0.5986],
        [0.3307, 0.6693],
        [0.3975, 0.6025],
        [0.2504, 0.7496],
        [0.3626, 0.6374],
        [0.2863, 0.7137],
        [0.3218, 0.6782],
        [0.3292, 0.6708],
        [0.2755, 0.7245],
        [0.2911, 0.7089],
        [0.3414, 0.6586],
        [0.3046, 0.6954],
        [0.2442, 0.7558],
        [0.3682, 0.6318]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 44 [   0/390]  Loss: 0.08213 (0.0821)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 1.843e-03
Train: 44 [  50/390]  Loss: 0.1111 (0.0724)  Acc@1: 96.8750 (97.7022)  Acc@5: 100.0000 (99.9694)LR: 1.843e-03
Train: 44 [ 100/390]  Loss: 0.04757 (0.0729)  Acc@1: 98.4375 (97.4783)  Acc@5: 100.0000 (99.9691)LR: 1.843e-03
Train: 44 [ 150/390]  Loss: 0.02291 (0.0717)  Acc@1: 100.0000 (97.5786)  Acc@5: 100.0000 (99.9690)LR: 1.843e-03
Train: 44 [ 200/390]  Loss: 0.09611 (0.0721)  Acc@1: 95.3125 (97.5435)  Acc@5: 100.0000 (99.9767)LR: 1.843e-03
Train: 44 [ 250/390]  Loss: 0.07127 (0.0714)  Acc@1: 98.4375 (97.5286)  Acc@5: 100.0000 (99.9813)LR: 1.843e-03
Train: 44 [ 300/390]  Loss: 0.1220 (0.0700)  Acc@1: 95.3125 (97.5810)  Acc@5: 100.0000 (99.9792)LR: 1.843e-03
Train: 44 [ 350/390]  Loss: 0.01471 (0.0696)  Acc@1: 100.0000 (97.6051)  Acc@5: 100.0000 (99.9777)LR: 1.843e-03
Train: 44 [ 390/390]  Loss: 0.07810 (0.0696)  Acc@1: 95.0000 (97.5840)  Acc@5: 100.0000 (99.9800)LR: 1.843e-03
train_acc 97.584000
Valid: 44 [   0/390]  Loss: 0.3190 (0.319)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 44 [  50/390]  Loss: 0.7387 (0.473)  Acc@1: 78.1250 (87.5919)  Acc@5: 98.4375 (99.5098)
Valid: 44 [ 100/390]  Loss: 0.7568 (0.479)  Acc@1: 81.2500 (87.7166)  Acc@5: 100.0000 (99.3967)
Valid: 44 [ 150/390]  Loss: 0.2121 (0.474)  Acc@1: 93.7500 (87.9346)  Acc@5: 100.0000 (99.3998)
Valid: 44 [ 200/390]  Loss: 0.5020 (0.475)  Acc@1: 85.9375 (87.8654)  Acc@5: 100.0000 (99.4092)
Valid: 44 [ 250/390]  Loss: 0.2332 (0.474)  Acc@1: 92.1875 (87.9358)  Acc@5: 98.4375 (99.4024)
Valid: 44 [ 300/390]  Loss: 0.4379 (0.472)  Acc@1: 87.5000 (87.9257)  Acc@5: 98.4375 (99.4082)
Valid: 44 [ 350/390]  Loss: 0.6048 (0.463)  Acc@1: 85.9375 (88.0164)  Acc@5: 98.4375 (99.4168)
Valid: 44 [ 390/390]  Loss: 0.2080 (0.460)  Acc@1: 92.5000 (88.1320)  Acc@5: 100.0000 (99.4240)
valid_acc 88.132000
epoch = 44   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2066, 0.7934],
        [0.2095, 0.7905],
        [0.1327, 0.8673],
        [0.2471, 0.7529],
        [0.3258, 0.6742],
        [0.3714, 0.6286],
        [0.4120, 0.5880],
        [0.5639, 0.4361],
        [0.4119, 0.5881],
        [0.2351, 0.7649],
        [0.6013, 0.3987],
        [0.4106, 0.5894],
        [0.4891, 0.5109],
        [0.1204, 0.8796]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3994, 0.6006],
        [0.3289, 0.6711],
        [0.4031, 0.5969],
        [0.2522, 0.7478],
        [0.3603, 0.6397],
        [0.2830, 0.7170],
        [0.3164, 0.6836],
        [0.3272, 0.6728],
        [0.2711, 0.7289],
        [0.2867, 0.7133],
        [0.3369, 0.6631],
        [0.3047, 0.6953],
        [0.2391, 0.7609],
        [0.3671, 0.6329]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 45 [   0/390]  Loss: 0.04623 (0.0462)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.587e-03
Train: 45 [  50/390]  Loss: 0.03170 (0.0651)  Acc@1: 100.0000 (98.0086)  Acc@5: 100.0000 (100.0000)LR: 1.587e-03
Train: 45 [ 100/390]  Loss: 0.1193 (0.0632)  Acc@1: 95.3125 (97.8496)  Acc@5: 100.0000 (99.9845)LR: 1.587e-03
Train: 45 [ 150/390]  Loss: 0.09534 (0.0646)  Acc@1: 98.4375 (97.8270)  Acc@5: 100.0000 (99.9897)LR: 1.587e-03
Train: 45 [ 200/390]  Loss: 0.03972 (0.0641)  Acc@1: 100.0000 (97.8389)  Acc@5: 100.0000 (99.9922)LR: 1.587e-03
Train: 45 [ 250/390]  Loss: 0.07959 (0.0643)  Acc@1: 96.8750 (97.8337)  Acc@5: 100.0000 (99.9938)LR: 1.587e-03
Train: 45 [ 300/390]  Loss: 0.07669 (0.0646)  Acc@1: 95.3125 (97.8509)  Acc@5: 100.0000 (99.9948)LR: 1.587e-03
Train: 45 [ 350/390]  Loss: 0.06988 (0.0647)  Acc@1: 96.8750 (97.8276)  Acc@5: 100.0000 (99.9955)LR: 1.587e-03
Train: 45 [ 390/390]  Loss: 0.1987 (0.0644)  Acc@1: 97.5000 (97.8440)  Acc@5: 100.0000 (99.9960)LR: 1.587e-03
train_acc 97.844000
Valid: 45 [   0/390]  Loss: 0.5957 (0.596)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 45 [  50/390]  Loss: 0.3644 (0.421)  Acc@1: 87.5000 (88.8787)  Acc@5: 100.0000 (99.3566)
Valid: 45 [ 100/390]  Loss: 0.2183 (0.427)  Acc@1: 93.7500 (88.5520)  Acc@5: 98.4375 (99.4585)
Valid: 45 [ 150/390]  Loss: 0.4010 (0.439)  Acc@1: 89.0625 (88.1933)  Acc@5: 98.4375 (99.5240)
Valid: 45 [ 200/390]  Loss: 0.1599 (0.439)  Acc@1: 92.1875 (88.2618)  Acc@5: 100.0000 (99.5336)
Valid: 45 [ 250/390]  Loss: 0.3522 (0.446)  Acc@1: 87.5000 (88.1163)  Acc@5: 100.0000 (99.5393)
Valid: 45 [ 300/390]  Loss: 0.2658 (0.447)  Acc@1: 90.6250 (88.1385)  Acc@5: 100.0000 (99.5484)
Valid: 45 [ 350/390]  Loss: 0.4416 (0.449)  Acc@1: 87.5000 (88.1588)  Acc@5: 100.0000 (99.5148)
Valid: 45 [ 390/390]  Loss: 0.4340 (0.444)  Acc@1: 85.0000 (88.2440)  Acc@5: 97.5000 (99.5200)
valid_acc 88.244000
epoch = 45   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1982, 0.8018],
        [0.2039, 0.7961],
        [0.1292, 0.8708],
        [0.2449, 0.7551],
        [0.3304, 0.6696],
        [0.3732, 0.6268],
        [0.4165, 0.5835],
        [0.5700, 0.4300],
        [0.4143, 0.5857],
        [0.2317, 0.7683],
        [0.6130, 0.3870],
        [0.4143, 0.5857],
        [0.4970, 0.5030],
        [0.1200, 0.8800]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3932, 0.6068],
        [0.3237, 0.6763],
        [0.3992, 0.6008],
        [0.2498, 0.7502],
        [0.3550, 0.6450],
        [0.2796, 0.7204],
        [0.3121, 0.6879],
        [0.3259, 0.6741],
        [0.2695, 0.7305],
        [0.2803, 0.7197],
        [0.3334, 0.6666],
        [0.3034, 0.6966],
        [0.2369, 0.7631],
        [0.3636, 0.6364]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 46 [   0/390]  Loss: 0.07149 (0.0715)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [  50/390]  Loss: 0.02027 (0.0630)  Acc@1: 100.0000 (97.7328)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [ 100/390]  Loss: 0.09155 (0.0669)  Acc@1: 96.8750 (97.6021)  Acc@5: 100.0000 (99.9691)LR: 1.377e-03
Train: 46 [ 150/390]  Loss: 0.1256 (0.0643)  Acc@1: 96.8750 (97.6718)  Acc@5: 100.0000 (99.9793)LR: 1.377e-03
Train: 46 [ 200/390]  Loss: 0.01945 (0.0636)  Acc@1: 98.4375 (97.7690)  Acc@5: 100.0000 (99.9845)LR: 1.377e-03
Train: 46 [ 250/390]  Loss: 0.04483 (0.0637)  Acc@1: 98.4375 (97.7963)  Acc@5: 100.0000 (99.9813)LR: 1.377e-03
Train: 46 [ 300/390]  Loss: 0.03888 (0.0650)  Acc@1: 98.4375 (97.7004)  Acc@5: 100.0000 (99.9844)LR: 1.377e-03
Train: 46 [ 350/390]  Loss: 0.05313 (0.0660)  Acc@1: 98.4375 (97.6451)  Acc@5: 100.0000 (99.9866)LR: 1.377e-03
Train: 46 [ 390/390]  Loss: 0.02047 (0.0640)  Acc@1: 100.0000 (97.7400)  Acc@5: 100.0000 (99.9880)LR: 1.377e-03
train_acc 97.740000
Valid: 46 [   0/390]  Loss: 0.5007 (0.501)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 46 [  50/390]  Loss: 0.7503 (0.458)  Acc@1: 84.3750 (88.0821)  Acc@5: 96.8750 (99.4485)
Valid: 46 [ 100/390]  Loss: 0.5581 (0.458)  Acc@1: 89.0625 (87.6856)  Acc@5: 100.0000 (99.5514)
Valid: 46 [ 150/390]  Loss: 0.3480 (0.454)  Acc@1: 90.6250 (88.0174)  Acc@5: 100.0000 (99.5757)
Valid: 46 [ 200/390]  Loss: 0.5583 (0.452)  Acc@1: 84.3750 (88.0442)  Acc@5: 98.4375 (99.6035)
Valid: 46 [ 250/390]  Loss: 0.2807 (0.446)  Acc@1: 95.3125 (88.2844)  Acc@5: 98.4375 (99.6016)
Valid: 46 [ 300/390]  Loss: 0.1823 (0.445)  Acc@1: 95.3125 (88.3046)  Acc@5: 100.0000 (99.6003)
Valid: 46 [ 350/390]  Loss: 0.1451 (0.443)  Acc@1: 93.7500 (88.4571)  Acc@5: 100.0000 (99.5682)
Valid: 46 [ 390/390]  Loss: 0.2555 (0.445)  Acc@1: 95.0000 (88.3720)  Acc@5: 100.0000 (99.5360)
valid_acc 88.372000
epoch = 46   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1949, 0.8051],
        [0.2000, 0.8000],
        [0.1272, 0.8728],
        [0.2433, 0.7567],
        [0.3336, 0.6664],
        [0.3761, 0.6239],
        [0.4210, 0.5790],
        [0.5786, 0.4214],
        [0.4204, 0.5796],
        [0.2319, 0.7681],
        [0.6224, 0.3776],
        [0.4200, 0.5800],
        [0.5074, 0.4926],
        [0.1187, 0.8813]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3947, 0.6053],
        [0.3166, 0.6834],
        [0.3990, 0.6010],
        [0.2450, 0.7550],
        [0.3513, 0.6487],
        [0.2768, 0.7232],
        [0.3110, 0.6890],
        [0.3249, 0.6751],
        [0.2644, 0.7356],
        [0.2773, 0.7227],
        [0.3315, 0.6685],
        [0.3037, 0.6963],
        [0.2321, 0.7679],
        [0.3619, 0.6381]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 47 [   0/390]  Loss: 0.04508 (0.0451)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [  50/390]  Loss: 0.01655 (0.0441)  Acc@1: 100.0000 (98.4069)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 100/390]  Loss: 0.04879 (0.0514)  Acc@1: 98.4375 (98.1590)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 150/390]  Loss: 0.02748 (0.0537)  Acc@1: 100.0000 (98.1374)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 200/390]  Loss: 0.04547 (0.0550)  Acc@1: 98.4375 (98.1110)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 250/390]  Loss: 0.02791 (0.0571)  Acc@1: 100.0000 (98.0142)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 300/390]  Loss: 0.02198 (0.0559)  Acc@1: 100.0000 (98.0793)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 350/390]  Loss: 0.01281 (0.0565)  Acc@1: 100.0000 (98.0235)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 390/390]  Loss: 0.08980 (0.0566)  Acc@1: 92.5000 (98.0080)  Acc@5: 100.0000 (99.9960)LR: 1.213e-03
train_acc 98.008000
Valid: 47 [   0/390]  Loss: 0.3513 (0.351)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)
Valid: 47 [  50/390]  Loss: 0.2655 (0.446)  Acc@1: 90.6250 (88.2966)  Acc@5: 100.0000 (99.4792)
Valid: 47 [ 100/390]  Loss: 0.6210 (0.459)  Acc@1: 87.5000 (87.9332)  Acc@5: 100.0000 (99.4585)
Valid: 47 [ 150/390]  Loss: 0.6732 (0.455)  Acc@1: 79.6875 (87.9553)  Acc@5: 98.4375 (99.5137)
Valid: 47 [ 200/390]  Loss: 0.4944 (0.467)  Acc@1: 79.6875 (87.6632)  Acc@5: 100.0000 (99.5103)
Valid: 47 [ 250/390]  Loss: 0.3451 (0.457)  Acc@1: 93.7500 (87.9607)  Acc@5: 100.0000 (99.4584)
Valid: 47 [ 300/390]  Loss: 0.4652 (0.457)  Acc@1: 84.3750 (88.0139)  Acc@5: 100.0000 (99.4861)
Valid: 47 [ 350/390]  Loss: 0.1253 (0.454)  Acc@1: 95.3125 (88.1588)  Acc@5: 100.0000 (99.4881)
Valid: 47 [ 390/390]  Loss: 0.2116 (0.453)  Acc@1: 87.5000 (88.2240)  Acc@5: 100.0000 (99.5080)
valid_acc 88.224000
epoch = 47   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1889, 0.8111],
        [0.1970, 0.8030],
        [0.1238, 0.8762],
        [0.2379, 0.7621],
        [0.3308, 0.6692],
        [0.3744, 0.6256],
        [0.4190, 0.5810],
        [0.5829, 0.4171],
        [0.4266, 0.5734],
        [0.2301, 0.7699],
        [0.6285, 0.3715],
        [0.4266, 0.5734],
        [0.5126, 0.4874],
        [0.1186, 0.8814]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3936, 0.6064],
        [0.3177, 0.6823],
        [0.3940, 0.6060],
        [0.2391, 0.7609],
        [0.3500, 0.6500],
        [0.2726, 0.7274],
        [0.3123, 0.6877],
        [0.3212, 0.6788],
        [0.2637, 0.7363],
        [0.2812, 0.7188],
        [0.3278, 0.6722],
        [0.2981, 0.7019],
        [0.2289, 0.7711],
        [0.3590, 0.6410]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 48 [   0/390]  Loss: 0.02285 (0.0229)  Acc@1: 100.0000 (100.0000)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [  50/390]  Loss: 0.05862 (0.0598)  Acc@1: 96.8750 (97.7022)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 100/390]  Loss: 0.07403 (0.0554)  Acc@1: 96.8750 (98.0043)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 150/390]  Loss: 0.03958 (0.0539)  Acc@1: 98.4375 (98.0857)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 200/390]  Loss: 0.03162 (0.0565)  Acc@1: 100.0000 (98.0410)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 250/390]  Loss: 0.07230 (0.0561)  Acc@1: 95.3125 (98.0266)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 300/390]  Loss: 0.09224 (0.0548)  Acc@1: 96.8750 (98.0897)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 350/390]  Loss: 0.1456 (0.0564)  Acc@1: 93.7500 (98.0324)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 390/390]  Loss: 0.02310 (0.0568)  Acc@1: 100.0000 (98.0160)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
train_acc 98.016000
Valid: 48 [   0/390]  Loss: 0.5554 (0.555)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)
Valid: 48 [  50/390]  Loss: 0.07042 (0.437)  Acc@1: 98.4375 (88.8787)  Acc@5: 100.0000 (99.4792)
Valid: 48 [ 100/390]  Loss: 0.4891 (0.452)  Acc@1: 85.9375 (88.2890)  Acc@5: 98.4375 (99.3967)
Valid: 48 [ 150/390]  Loss: 0.2845 (0.455)  Acc@1: 92.1875 (88.3278)  Acc@5: 100.0000 (99.3895)
Valid: 48 [ 200/390]  Loss: 0.4589 (0.454)  Acc@1: 85.9375 (88.1841)  Acc@5: 100.0000 (99.4481)
Valid: 48 [ 250/390]  Loss: 0.4903 (0.463)  Acc@1: 92.1875 (88.0478)  Acc@5: 100.0000 (99.4397)
Valid: 48 [ 300/390]  Loss: 0.7524 (0.467)  Acc@1: 79.6875 (88.0035)  Acc@5: 100.0000 (99.4601)
Valid: 48 [ 350/390]  Loss: 0.2607 (0.465)  Acc@1: 92.1875 (87.9585)  Acc@5: 100.0000 (99.4703)
Valid: 48 [ 390/390]  Loss: 0.3986 (0.467)  Acc@1: 87.5000 (87.9480)  Acc@5: 100.0000 (99.4480)
valid_acc 87.948000
epoch = 48   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1818, 0.8182],
        [0.1905, 0.8095],
        [0.1192, 0.8808],
        [0.2331, 0.7669],
        [0.3327, 0.6673],
        [0.3747, 0.6253],
        [0.4196, 0.5804],
        [0.5885, 0.4115],
        [0.4298, 0.5702],
        [0.2255, 0.7745],
        [0.6334, 0.3666],
        [0.4341, 0.5659],
        [0.5193, 0.4807],
        [0.1173, 0.8827]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3952, 0.6048],
        [0.3147, 0.6853],
        [0.3896, 0.6104],
        [0.2373, 0.7627],
        [0.3485, 0.6515],
        [0.2630, 0.7370],
        [0.3110, 0.6890],
        [0.3182, 0.6818],
        [0.2603, 0.7397],
        [0.2762, 0.7238],
        [0.3181, 0.6819],
        [0.2972, 0.7028],
        [0.2238, 0.7762],
        [0.3556, 0.6444]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 49 [   0/390]  Loss: 0.08290 (0.0829)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [  50/390]  Loss: 0.03477 (0.0517)  Acc@1: 100.0000 (98.1924)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [ 100/390]  Loss: 0.07606 (0.0539)  Acc@1: 95.3125 (98.1126)  Acc@5: 100.0000 (99.9845)LR: 1.024e-03
Train: 49 [ 150/390]  Loss: 0.03672 (0.0575)  Acc@1: 100.0000 (97.9615)  Acc@5: 100.0000 (99.9897)LR: 1.024e-03
Train: 49 [ 200/390]  Loss: 0.1028 (0.0560)  Acc@1: 95.3125 (98.0799)  Acc@5: 100.0000 (99.9922)LR: 1.024e-03
Train: 49 [ 250/390]  Loss: 0.03431 (0.0551)  Acc@1: 100.0000 (98.1076)  Acc@5: 100.0000 (99.9938)LR: 1.024e-03
Train: 49 [ 300/390]  Loss: 0.04356 (0.0547)  Acc@1: 100.0000 (98.1676)  Acc@5: 100.0000 (99.9948)LR: 1.024e-03
Train: 49 [ 350/390]  Loss: 0.03953 (0.0559)  Acc@1: 98.4375 (98.1081)  Acc@5: 100.0000 (99.9955)LR: 1.024e-03
Train: 49 [ 390/390]  Loss: 0.06101 (0.0558)  Acc@1: 97.5000 (98.1200)  Acc@5: 100.0000 (99.9960)LR: 1.024e-03
train_acc 98.120000
Valid: 49 [   0/390]  Loss: 0.4401 (0.440)  Acc@1: 89.0625 (89.0625)  Acc@5: 98.4375 (98.4375)
Valid: 49 [  50/390]  Loss: 0.7726 (0.469)  Acc@1: 84.3750 (87.9902)  Acc@5: 98.4375 (99.2953)
Valid: 49 [ 100/390]  Loss: 0.3336 (0.475)  Acc@1: 92.1875 (87.8868)  Acc@5: 100.0000 (99.3967)
Valid: 49 [ 150/390]  Loss: 0.3778 (0.475)  Acc@1: 90.6250 (87.8622)  Acc@5: 100.0000 (99.4309)
Valid: 49 [ 200/390]  Loss: 0.7733 (0.473)  Acc@1: 84.3750 (87.8343)  Acc@5: 96.8750 (99.4636)
Valid: 49 [ 250/390]  Loss: 0.6424 (0.473)  Acc@1: 89.0625 (87.9607)  Acc@5: 100.0000 (99.4522)
Valid: 49 [ 300/390]  Loss: 0.2651 (0.463)  Acc@1: 93.7500 (88.1177)  Acc@5: 100.0000 (99.4653)
Valid: 49 [ 350/390]  Loss: 0.5034 (0.460)  Acc@1: 89.0625 (88.1321)  Acc@5: 100.0000 (99.4792)
Valid: 49 [ 390/390]  Loss: 0.4874 (0.459)  Acc@1: 85.0000 (88.1200)  Acc@5: 100.0000 (99.4800)
valid_acc 88.120000
epoch = 49   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1779, 0.8221],
        [0.1869, 0.8131],
        [0.1164, 0.8836],
        [0.2321, 0.7679],
        [0.3324, 0.6676],
        [0.3730, 0.6270],
        [0.4182, 0.5818],
        [0.5935, 0.4065],
        [0.4354, 0.5646],
        [0.2266, 0.7734],
        [0.6392, 0.3608],
        [0.4401, 0.5599],
        [0.5268, 0.4732],
        [0.1161, 0.8839]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3983, 0.6017],
        [0.3158, 0.6842],
        [0.3881, 0.6119],
        [0.2351, 0.7649],
        [0.3452, 0.6548],
        [0.2600, 0.7400],
        [0.3077, 0.6923],
        [0.3154, 0.6846],
        [0.2589, 0.7411],
        [0.2766, 0.7234],
        [0.3128, 0.6872],
        [0.2957, 0.7043],
        [0.2220, 0.7780],
        [0.3544, 0.6456]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
